S3AM: A Spectral-Similarity-Based Spatial Attention Module for Hyperspectral Image Classification

Abstract

Recently, hyperspectral image (HSI) classification based on deep learning methods has attracted growing attention and made great progress . Convolutional neural networks based models, especially the residual networks (ResNets), have become the architectures of choice for extracting the deep spectral-spatial features. However, there are generally some interfering pixels in the neighborhoods of the center pixel, which are unfavorable for the spectral-spatial feature extraction and will lead to a restraint classification performance. More important, the existing attention modules are weak in highlighting the effect of the center pixel for the spatial attention. To solve this issue, this article proposes a novel spectral-similarity-based spatial attention module (S 3 AM) to emphasize the relevant spatial areas in HSI. The S 3 AM adopts the weighted Euclidean and cosine distances to measure the spectral similarities between the center pixel and its neighborhoods. To alleviate the negative influence of the spectral variability, the full-band convolutional layers are deployed to reweight the bands for the robust spectral similarities. Both kinds of weighted spectral similarities are then fused adaptively to take their relative importance into full account. Finally, a scalable Gaussian activation function, which can suppress the interfering pixels dynamically, is installed to transform the spectral similarities into the appropriate spatial weights. The S 3 AM is integrated with the ResNet to build the S 3 AM-Net model, which is able to extract the discriminating spectral-spatial features. Experimental results on four public HSI datasets demonstrate the effectiveness of the proposed attention module and the outstanding classification performance of the S 3 AM-Net model.