Abstract

Accurate building segmentation from high-resolution aerial imagery remains challenging due to variations in building size, shape, background clutter, and class imbalance. This study proposes an Enhanced U-Net architecture for automatic building extraction using the Massachusetts Buildings Dataset and the Inria Aerial Image Labeling Dataset. The proposed model extends the conventional U-Net by incorporating a deeper encoder with Batch Normalization, Attention Gates (AGs) in decoder skip connections, and an Atrous Spatial Pyramid Pooling (ASPP) module for multi-scale contextual feature extraction. Different ASPP dilation-rate configurations were investigated, with dilation rates of 3, 6, and 9 yielding the best performance. To handle the high spatial resolution of the imagery, a patch-based training strategy with 50% overlap was adopted, using patch sizes of 256 × 256 and 512 × 512 for the Massachusetts and Inria datasets, respectively. The model was optimized using a hybrid Binary Cross-Entropy (BCE) and Dice loss function to balance pixel-level classification and region-overlap accuracy. Experimental results demonstrate that the proposed Enhanced U-Net outperformed U-Net, DeepLabv3+, and HRNet. On the Massachusetts Buildings Dataset, it achieved an IoU of 0.737 and an F1-score of 0.848. On the Inria dataset, it achieved an IoU of 0.799 and an F1-score of 0.888. These results demonstrate the effectiveness and generalization capability of the proposed architecture for aerial building segmentation.

References

• Ayala et al., "A Deep Learning Approach to an Enhanced Building Footprint and Road Detection in High-Resolution Satellite Imagery," Remote Sensing, vol. 13, no. 16, 2021. .
• Wagner et al., "U-Net-Id, an Instance Segmentation Model for Building Extraction from Satellite Images—Case Study in the Joanópolis City, Brazil," Remote Sensing, vol. 12, no. 10, 2020.
• R. Olaf, F. Philipp and B. Thomas, "U-Net: Convolutional Networks for Biomedical Image Segmentation," Computer Vision and Pattern Recognition, vol. 9351, p. 234–241, 2015.
• Shelhamer et al, "Fully Convolutional Networks for Semantic Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640-651, 2016.
• Zhuokun et al., "Deep Learning Segmentation and Classification for Urban Village Using a Worldview Satellite Image Based on U-Net," Remote Sensing, vol. 12, no. 1574, pp. 2-17, 2000.
• W. Alsabhan, T. Alotaiby and B. Dudin, "Detecting Buildings and Nonbuildings from Satellite Images Using U-Net," Hindawi, vol. 2022, pp. 1-13, 2022.
• Najjaj et al., "Deep Learning Approach for Uban Mapping," in The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Gottingen, 2022.
• E. Wang and D. Wang, "Using U-Net to Detect Buildings in Satellite Images," Journal of Computer and Communication, vol. 10, pp. 132-138., 2022.
• E. Irwansyah, Y. Heryadi and A. A. S. Gunawan, "Semantic Image Segmentation for Building Detection in Urban Area with Aerial Photograph Image using U-Net Models," IEEE Explore, pp. 48-51, 2020.
• Weijia et al., "Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data," Remote Sensing, vol. 11, no. 4, 2019.
• J. Shunping, W. Shiqing and L. Meng, "Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set," IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, vol. 57, no. 1, pp. 574-586, 2018.
• S. Batuhan and D. Z. Seker, "A Residual-Inception U-Net (RIU-Net) Approach and Comparisons with U-Shaped CNN and Transformer Models for Building Segmentation from High-Resolution Satellite Images," Senors, vol. 22, no. 19, pp. 1-20, 2022.
• Wang et al., "Building extraction with vision transformer," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, no. 5625711, pp. 1-11, 2022.
• Wang et al., "Performance Analysis of Various EfficientNet Based U-Net++ Architecture for Automatic Building Extraction from High Resolution Satellite Images," Springe, vol. 376, pp. 385-399, 2024.
• Zhou et al., "Enhanced Building Extraction via STMC-UNet: Integrating Super Token Transformer and Multi-scale Convolution," Signal, Image and Video Processing, vol. 19, no. 1019, 2025.
• S. Ekiz and U. Acar, "Improving building extraction from high-resolution aerial images: Error correction and performance enhancement using deep learning on the Inria dataset," Sage Journals, vol. 108(I), pp. 1-22, 2025.
• Hui et al., "An improved DeepLabv3+ lightweight network for remote-sensing image semantic segmentation," Springer: Complex & Intelligent Systems, vol. 10, pp. 2839-2849, 2024.
• R. C. Satyaki and G. Mirzaei, "Hybridization of Attention UNet with Repeated Atrous Spatial Pyramid Pooling for Improved Brain Tumour Segmentation," in 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), Houston, TX, USA, 2025.
• Wang et al., "SAR-U-Net: Squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in Computed Tomography," Elsevier, vol. 208, 2021.
• Wang et al., "Improved Unet model for brain tumor image segmentation based on ASPP-coordinate attention mechanism," in 5th International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE, Wenzhou, China, 2024.
• Xiangsuo et al., "Improved U-Net Remote Sensing Classification Algorithm Fusing Attention and Multiscale Features," Remote Sensing, vol. 14, no. 15, pp. 1-24, 2022.
• Jovial et al., "Attention-Guided Residual U-Net with SE Connection and ASPP for Watershed-Based Cell Segmentation in Microscopy Images," in National Library of Medeicine, 2025.
• I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press , 2016.
• Ozan et al., "Attention U-Net: Learning Where to Look for the Pancreas," Conrnell University, 2018.
• B. Ashwath, "Kaggle," 2020. [Online]. Available: https://www.kaggle.com/datasets/balraj98/massachusetts-buildings-dataset. [Accessed 15 November 2024].
• Maggiori et al., "The Inria Aerial Image Labeling Benchmark," in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Texas, United States, 2017.
• R. Sagar, "Inria Aerial Image Labeling Dataset," Kaggle, 2021. [Online]. Available: https://www.kaggle.com/datasets/sagar100rathod/inria-aerial-image-labeling-dataset. [Accessed 20 May 2026].
• N. Joseph, "Enhanced U-Net for Building Segmentation," GitHub, 12 26 2026. [Online]. Available: https://github.com/joejoedat-collab/enhanced-unet-building-segmentation. [Accessed 26 January 2026].