集成空间变换结构与深度残差网络的遥感影像场景分类方法

孟亦菲; 郑贵洲; 冀炜臻

doi:10.3799/dqkx.2021.218

Volume 48 Issue 9

Sep. 2023

Turn off MathJax

Article Contents

Article Navigation > Earth Science > 2023 > 48(9): 3526-3538

Meng Yifei, Zheng Guizhou, Ji Weizhen, 2023. Remote Sensing Image Scene Classification Method Integrating Spatial Transformation Structure and Depth Residual Network. Earth Science, 48(9): 3526-3538. doi: 10.3799/dqkx.2021.218

Citation:

Meng Yifei, Zheng Guizhou, Ji Weizhen, 2023. Remote Sensing Image Scene Classification Method Integrating Spatial Transformation Structure and Depth Residual Network. Earth Science, 48(9): 3526-3538. doi: 10.3799/dqkx.2021.218

Citation:

PDF( 4264 KB)

Remote Sensing Image Scene Classification Method Integrating Spatial Transformation Structure and Depth Residual Network

doi: 10.3799/dqkx.2021.218

Meng Yifei^{1
,
,},
Zheng Guizhou^{1
,
,
,},
Ji Weizhen²

1.
School of Geography and Information Engineering, China University of Geosciences, Wuhan 430078, China
2.
School of Architectural and Surveying and Mapping Engineering, Jiangxi University of Science and Technology, Ganzhou 341000, China

Received Date: 2021-07-07
Available Online: 2023-10-07
Publish Date: 2023-09-25

Abstract

Abstract

In order to solve the problem that the remote sensing image with small sample set can easily lead to the over-fitting of the training model and the low classification accuracy caused by the spatial invariance of convolution neural network in remote sensing image scene classification, a high-resolution remote sensing image scene classification algorithm based on spatial transformation network and transfer learning is proposed. Firstly, the ImageNet dataset is used to train the deep residual network ResNet101 to obtain the pre-training model, and the training efficiency of the model is improved through knowledge transfer. Then, the spatial transformation structure is embedded in the model, so that the model can actively transform the feature mapping in space and improve the robustness of the model. Finally, the Dropout layer is added to the model to reduce the probability of over-fitting of the model. This method is verified on two high-score remote sensing image data sets of AID and NWPU-RESISC45, and the classification accuracy of 94.30% and 93.63% is achieved in the case of only 20% training samples. The experimental results show that the improved model has better feature extraction ability and better classification results for misclassification scenarios.
- deep learning,
- residual network,
- spatial transformation networks,
- transfer learning,
- scene classification,
- remote sensing

FullText(HTML)

References(46)

References

Berman, M., Triki, A. R., Blaschko, M. B., 2018. The Lovasz-Softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-Over-Union Measure in Neural Networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City. https://doi.org/10.1109/CVPR.2018.00464

Cheng, G., Guo, L., Zhao, T. Y., et al., 2013. Automatic Landslide Detection from Remote-Sensing Imagery Using a Scene Classification Method Based on BoVW and pLSA. International Journal of Remote Sensing, 34(1): 45-59. https://doi.org/10.1080/01431161.2012.705443

Cheng, G., Han, J. W., Lu, X. Q., 2017. Remote Sensing Image Scene Classification: Benchmark and State of the Art. Proceedings of the IEEE, 105(10): 1865-1883. https://doi.org/10.1109/JPROC.2017.2675998

Cheng, G., Yang, C. Y., Yao, X. W., et al., 2018. When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNS. IEEE Transactions on Geoscience and Remote Sensing, 56(5): 2811-2821. https://doi.org/10.1109/TGRS.2017.2783902

Cheng, G. X., Niu, R. Q., Zhang, K. X., et al., 2018. Opencast Mining Area Recognition in High-Resolution Remote Sensing Images Using Convolutional Neural Networks. Earth Science, 43(S2): 256-262 (in Chinese with English abstract).

Dalal, N., Triggs, B., 2005. Histograms of Oriented Gradients for Human Detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego. https://doi.org/10.1109/CVPR.2005.177

Donahue, J., Jia, Y. Q., Vinyals, O., et al., 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. Proceedings of the 31st International Conference on Machine Learning, Beijing. https://doi.org/10.5555/3044805.3044879

Han, X., Zhang, Z. Y., Ding, N., et al., 2021. Pre-Trained Models: Past, Present and Future. AI Open, 2: 225-250. https://doi.org/10.1016/j.aiopen.2021.08.002

He, K. M., Zhang, X. Y., Ren, S. Q., et al., 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas. https://doi.org/10.1109/CVPR.2016.90

He, N. J., Fang, L. Y., Li, S. T., et al., 2018. Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling. IEEE Transactions on Geoscience and Remote Sensing, 56(12): 6899-6910. https://doi.org/10.1109/TGRS.2018.2845668

He, X., Chen, Y. S., 2019. Optimized Input for CNN-Based Hyperspectral Image Classification Using Spatial Transformer Network. IEEE Geoscience and Remote Sensing Letters, 16(12): 1884-1888. https://doi.org/10.1109/LGRS.2019.2911322

Jaderberg, M., Simonyan, K., Zisserman, A., et al., 2015. Spatial Transformer Networks. arXiv, 1506.02025. https://arxiv.org/abs/1506.02025

Jia, Y. Q., Shelhamer, E., Donahue, J., et al., 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. Proceedings of the 22nd ACM international conference on Multimedia, Orlando. https://doi.org/10.1145/2647868.2654889

Krizhevsky, A., Sutskever, I., Hinton, G. E., 2012. ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe. https://doi.org/10.5555/2999134.2999257

Li, D. R., Wang, M., Shen, X., et al., 2017. From Earth Observation Satellite to Earth Observation Brain. Geomatics and Information Science of Wuhan University, 42(2): 143-149 (in Chinese with English abstract).

Li, G. D., Zhang, C. J., Wang, M. K., et al., 2019. Transfer Learning Using Convolutional Neural Network for Scene Classification within High Resolution Remote Sensing Image. Science of Surveying and Mapping, 44(4): 116-123, 174 (in Chinese with English abstract).

Oliva, A., Torralba, A., 2001. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. International Journal of Computer Vision, 42(3): 145-175. https://doi.org/10.1023/A: 1011139631724 doi: 10.1023/A:1011139631724

Oquab, M., Bottou, L., Laptev, I., et al., 2014. Learning and Transferring Mid-Level Image Representations Using Convolutional Neural Networks. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus. https://doi.org/10.1109/CVPR.2014.222

Pan, S. J., Yang, Q., 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10): 1345-1359. https://doi.org/10.1109/TKDE.2009.191

Perronnin, F., Sánchez, J., Mensink, T., 2010. Improving the Fisher Kernel for Large-Scale Image Classification. European Conference on Computer Vision, Berlin. https://doi.org/10.1007/978-3-642-15561-1_11

Simonyan, K., Zisserman, A., 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. ArXiv, 1409.1556. https://arxiv.org/abs/1409.1556

Srinivas, A., Lin, T. Y., Parmar, N., et al., 2021. Bottleneck Transformers for Visual Recognition. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville. https://doi.org/10.1109/CVPR46437.2021.01625

Szegedy, C., Liu, W., Jia, Y. Q., et al., 2015. Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston. https://doi.org/10.1109/CVPR.2015.7298594

Văduva, C., Gavăt, I., Datcu, M., 2013. Latent Dirichlet Allocation for Spatial Analysis of Satellite Images. IEEE Transactions on Geoscience and Remote Sensing, 51(5): 2770-2786. https://doi.org/10.1109/TGRS.2012.2219314

van de Sande, K., Gevers, T., Snoek, C., 2010. Evaluating Color Descriptors for Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): 1582-1596. https://doi.org/10.1109/TPAMI.2009.154

Wallraven, C., Caputo, B., Graf, A., 2003. Recognition with Local Features: The Kernel Recipe. Proceedings Ninth IEEE International Conference on Computer Vision, Nice. https://doi.org/10.1109/ICCV.2003.1238351

Wang, R. C., 2018. Feature Learning and Patch Matching of Multispectral Images Based on Deep Neural Networks (Dissertation). Beijing University of Posts and Telecommunications, Beijing (in Chinese with English abstract).

Weiss, K., Khoshgoftaar, T. M., Wang, D. D., 2016. A Survey of Transfer Learning. Journal of Big Data, 3(1): 1-40. https://doi.org/10.1186/s40537-016-0043-6

Xia, G. S., Hu, J. W., Hu, F., et al., 2017. AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981. https://doi.org/10.1109/TGRS.2017.2685945

Xu, Y. Y., Li, Z. X., Xie, Z., et al., 2020. Prediction of Copper Mineralization Based on Semi-Supervised Neural Network. Earth Science, 45(12): 4563-4573 (in Chinese with English abstract).

Yang, Y., Newsam, S., 2008. Comparing SIFT Descriptors and Gabor Texture Features for Classification of Remote Sensed Imagery. 2008 15th IEEE International Conference on Image Processing, San Diego. https://doi.org/10.1109/ICIP.2008.4712139

Yang, Y., Newsam, S., 2013. Geographic Image Retrieval Using Local Invariant Features. IEEE Transactions on Geoscience and Remote Sensing, 51(2): 818-832. https://doi.org/10.1109/TGRS.2012.2205158

Yu, D. H., Zhang, B. M., Zhao, C., et al., 2020. Scene Classification of Remote Sensing Image Using Ensemble Convolutional Neural Network. Journal of Remote Sensing, 24(6): 717-727 (in Chinese with English abstract).

Yu, S. C., Yu, D. Q., Wang, L. C., et al., 2019. Remote Sensing Study of Dongting Lake Beach Changes before and after Operation of Three Gorges Reservoir. Earth Science, 44(12): 4275-4283 (in Chinese with English abstract).

Zhang, K., Hei, B. Q., Li, S. Y., et al., 2018. Complex Scene Classification of Remote Sensing Images Based on CNN. Remote Sensing for Land & Resources, 30(4): 49-55 (in Chinese with English abstract).

Zhang, W., Tang, P., Zhao, L. J., 2019. Remote Sensing Image Scene Classification Using CNN-CapsNet. Remote Sensing, 11(5): 494. https://doi.org/10.3390/rs11050494

Zuo, R. G., Peng, Y., Li, T., et al., 2021. Challenges of Geological Prospecting Big Data Mining and Integration Using Deep Learning Algorithms. Earth Science, 46(1): 350-358 (in Chinese with English abstract).

程国轩, 牛瑞卿, 张凯翔, 等, 2018. 基于卷积神经网络的高分遥感影像露天采矿场识别. 地球科学, 43(S2): 256-262. doi: 10.3799/dqkx.2018.987

李德仁, 王密, 沈欣, 等, 2017. 从对地观测卫星到对地观测脑. 武汉大学学报(信息科学版), 42(2): 143-149. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201702001.htm

李冠东, 张春菊, 王铭恺, 等, 2019. 卷积神经网络迁移的高分影像场景分类学习. 测绘科学, 44(4): 116-123, 174. https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201904021.htm

王瑞琛, 2018. 基于深层神经网络的异源图像特征学习及块匹配(硕士学位论文). 北京: 北京邮电大学.

徐永洋, 李孜轩, 谢忠, 等, 2020. 基于半监督神经网络的铜矿预测方法. 地球科学, 45(12): 4563-4573. doi: 10.3799/dqkx.2020.297

余东行, 张保明, 赵传, 等, 2020. 联合卷积神经网络与集成学习的遥感影像场景分类. 遥感学报, 24(6): 717-727. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB202006006.htm

余姝辰, 余德清, 王伦澈, 等, 2019. 三峡水库运行前后洞庭湖洲滩面积变化遥感认识. 地球科学, 44(12): 4275-4283. doi: 10.3799/dqkx.2019.182

张康, 黑保琴, 李盛阳, 等, 2018. 基于CNN模型的遥感图像复杂场景分类. 国土资源遥感, 30(4): 49-55. https://www.cnki.com.cn/Article/CJFDTOTAL-GTYG201804008.htm

左仁广, 彭勇, 李童, 等, 2021. 基于深度学习的地质找矿大数据挖掘与集成的挑战. 地球科学, 46(1): 350-358. doi: 10.3799/dqkx.2020.111

Relative Articles

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(12) / Tables(4)

Get Citation

PDF

XML

Article views (324) PDF downloads(18)

Remote Sensing Image Scene Classification Method Integrating Spatial Transformation Structure and Depth Residual Network

doi: 10.3799/dqkx.2021.218

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Proportional views

Export File

Citation

Format

Content