A Comprehensive Evaluation of Deep Vision Transformers for Road Extraction from Very-high-resolution Satellite Data

Abstract

Transformer-based semantic segmentation architectures excel in extracting road networks from very-high-resolution (VHR) satellite images due to their ability to capture global contextual information. Nonetheless, there is a gap in research regarding their comparative effectiveness, efficiency, and performance in extracting road networks from multicity VHR data. This study evaluates 11 transformer-based models on three publicly available datasets (DeepGlobe Road Extraction Dataset, SpaceNet-3 Road Network Detection Dataset, and Massachusetts Road Dataset) to assess their performance, efficiency, and complexity in mapping road networks from multicity VHR satellite images. The evaluated models include Unified Perceptual Parsing for Scene Understanding (UperNet) based on the Swin transformer (UperNet-SwinT), and Multi-path Vision Transformer (UperNet-MpViT), Twins transformer, Segmenter, SegFormer, K-Net based on SwinT, Mask2Former based on SwinT (Mask2Former-SwinT), TopFormer, UniFormer, and PoolFormer. Results showed that the models recorded mean F-scores (mF-score) ranging from 82.22% to 90.70% for the DeepGlobe dataset, 58.98% to 86.95% for the Massachusetts dataset, and 69.02% to 86.14% for the SpaceNet-3 dataset. Mask2Former-SwinT, UperNet-MpViT, and SegFormer were the top performers among the evaluated models. The Mask2Former, based on the SwinT, demonstrated a strong balance of high performance across different satellite image datasets and moderate computational efficiency. This investigation aids in selecting the most suitable model for extracting road networks from remote sensing data.Transformer-based semantic segmentation architectures excel in extracting road networks from very-high-resolution (VHR) satellite images due to their ability to capture global contextual information. Nonetheless, there is a gap in research regarding their comparative effectiveness, efficiency, and performance in extracting road networks from multicity VHR data. This study evaluates 11 transformer-based models on three publicly available datasets (DeepGlobe Road Extraction Dataset, SpaceNet-3 Road Network Detection Dataset, and Massachusetts Road Dataset) to assess their performance, efficiency, and complexity in mapping road networks from multicity VHR satellite images. The evaluated models include Unified Perceptual Parsing for Scene Understanding (UperNet) based on the Swin transformer (UperNet-SwinT), and Multi-path Vision Transformer (UperNet-MpViT), Twins transformer, Segmenter, SegFormer, K-Net based on SwinT, Mask2Former based on SwinT (Mask2Former-SwinT), TopFormer, UniFormer, and PoolFormer. Results showed that the models recorded mean F-scores (mF-score) ranging from 82.22% to 90.70% for the DeepGlobe dataset, 58.98% to 86.95% for the Massachusetts dataset, and 69.02% to 86.14% for the SpaceNet-3 dataset. Mask2Former-SwinT, UperNet-MpViT, and SegFormer were the top performers among the evaluated models. The Mask2Former, based on the SwinT, demonstrated a strong balance of high performance across different satellite image datasets and moderate computational efficiency. This investigation aids in selecting the most suitable model for extracting road networks from remote sensing data.