Self-Supervised Learning Driven Cross-Domain Feature Fusion Network for Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification faces significant challenges due to the high cost of acquiring labeled samples. To mitigate this, we propose SSCF-Net, a novel self-supervised learning driven cross-domain feature fusion Network. SSCF-Net uniquely leverages readily available labeled natural images (source domain) to aid HSI (target domain) classification by transfer learning. Specifically, we employ rotation-based self-supervision in the source domain to learn transferable features, which are then transferred to the HSI domain. Within SSCF-Net, we effectively fuse local and global features: local features are extracted by a jointly trained module combining VGG and two-dimensional long short-term memory networks (TD-LSTM) networks, while global features capturing long-range dependencies are learned via a Transformer model. Crucially, in the HSI domain, we further employ contrastive learning as a self-supervised strategy to maximally utilize the limited labeled data. Extensive experiments on three benchmark HSI datasets (Salinas, Indian Pines, WHU-Hi-LongKou) demonstrate that SSCF-Net significantly outperforms existing methods, validating its effectiveness in addressing the label scarcity problem. The code is available at https://github.com/6pangbo/SSCF-Net.