Geological Body Recognition Based on Multi-Modal Feature Fusion
-
摘要: 将深度学习技术应用于地质填图，挖掘不同模态数据深层次信息，可以实现更为准确的地质填图.提出了一种基于多模态特征融合的地质体识别方法，综合考虑地球物理、地球化学数据和遥感影像数据，该方法先利用深度神经网络和卷积神经网络分别提取这两种不同模态数据的特征，然后进行特征拼接得到多模态特征，最后利用全连接神经网络进行特征融合完成地质体分类.交叉验证结果表明，提出的多模态特征融合方法比仅使用地球物理地球化学数据或遥感影像数据的深度学习方法相比有明显的优势，在分类准确率上分别提升了14.08%和2.79%，证明了该方法可以实现更为精准的地质体识别，进而更好地辅助地质填图.Abstract: Applying deep learning technology to geological mapping to mine the deep-level in formation of different modal data, so as to achieve more accurate geological mapping. Considering the geophysical and geochemical data and remote sensing image data, in this paper it proposes a geological body recognition method based on multi-modal feature fusion. Firstly, Using deep neural network and convolution neural network to extract the features of the two different modal data and then performs feature splicing to obtain multi-modal features, finally, the fully connected neural network is used for feature fusion to complete the geological body classification. The cross-validation results show that the proposed multi-modal feature fusion method has obvious advantages compared with the deep learning methods using geophysical and geochemical data or remote sensing image data along, and the classification accuracy rate is increased by 14.08% and 2.79%. This result proves that this method can realize more accurate geological body identification, and then better assist geological mapping.
-
图 1 多模态特征融合地质体识别模型整体结构
Fig. 1. The overall structure of multi-modal feature fusion geological body recognition model
图 4 卷积层数对比实验结果
a.不同卷积层数ACC与F1值；b.不同卷积层数预测时间与F1值
Fig. 4. Convolution layer number comparison experiment results
图 5 研究区局部实测图与预测图
a.研究区局部实测图；b.研究区局部预测图
Fig. 5. Part of the measured map and prediction map of the study area
表 1 PRB采样点及样本点数量
Table 1. The number of PRB sampling points and PRB sample points
地质体 PRB采样点 PRB样本点 K1b流纹质晶玻屑凝灰岩 13 5 200 J3mn安山质熔结角砾岩晶屑凝灰岩 14 5 600 J3m流纹质角砾晶屑玻屑凝灰岩 15 6 000 J3m流纹质晶玻屑凝灰岩 16 6 400 K1b流纹质角砾岩熔岩 17 6 800 J3m石泡流纹岩 17 6 800 J3mn角闪安山岩 18 7 200 J3m含角砾流纹岩 20 8 000 Qh1eol土黄色风成砂 21 8 400 J3m流纹斑岩 24 9 600 
下载: 导出CSV
表 2 PRB采样点信息
Table 2. PRB sampling point information
GEOPOINT 采样点类型 XX YY 填图单位+岩性 色标 DJ126 分段路线 325 660.43 4 976 502.71 ξπ正长斑岩 255, 0, 0 DK111 B标本 326 438.48 4 961 608.46 λπ流纹斑岩 255, 0, 0 DT534 岩性控制点 310 839.70 4 956 034.54 ηοπK1石英二长斑岩 255, 64, 25 DL111 b薄片 324 676.34 4 956 893.87 δμ闪长玢岩 0, 255, 255 DS352 岩性控制点 310 212.49 4 962 644.40 Qp3w乌尔吉组风成砂土 255, 255, 128 DJ112 分段路线 329 900.84 4 975 344.51 Qh3eol风成砂土 255, 255, 181 DP049 分段路线 317 272.82 4 957 868.69 P3l岩屑细砂岩 242, 216, 67 DJ541 岩性控制点 326 278.26 4 971 615.54 P3l片岩 242, 216, 81 DJ014 B标本 330 298.39 4 968 650.78 P3l构造角砾岩 105, 91, 7 DO5251 分段路线 319 140.89 4 965 597.51 J3m流纹质浆屑角砾凝灰岩 178, 254, 242
下载: 导出CSV
表 3 地物地化数据及遥感影像数据具体类别
Table 3. Specific categories of geophysical data, geochemical data and remote sensing image data
数据类型 分辨率 波段/元素数 1∶20万地球化学 5 m×5 m 39 1∶5万地球化学 5 m×5 m 12 地面高精度磁测 5 m×5 m 1 高分三号卫星 10 m×10 m 3 高分一号 & 资源三号卫星 2 m×2 m 3 数字高程模型 30 m×30 m 1 Landsat8卫星 30 m×30 m 3*4
下载: 导出CSV
表 4 不同遥感影像数据组合对比实验结果
Table 4. Comparison of experimental results of different remote sensing image data combinations
模型名称 数据集 ACC F1-score 卷积神经网络 GF-3+GF-1+ZY-3+Landsat8 91.05% 91.09% 卷积神经网络 GF-1+ZY-3+Landsat8+DEM 92.40% 92.59% 卷积神经网络 GF-3+GF-1+ZY-3+Landsat8+DEM 93.42% 93.52%
下载: 导出CSV
表 5 不同遥感影像数据范围的模型参数量、计算及预测时间
Table 5. The amount of model parameters, calculation and prediction time for different remote sensing image data ranges
遥感影像
数据范围参数量 训练时间
(s/轮)预测时间
(s)5×5 197 223 1 62 10×10 516 455 2 97 15×15 778 599 3 139
下载: 导出CSV
表 6 不同遥感影像数据范围对比实验结果
Table 6. Comparison of experimental results of different remote sensing image data ranges
遥感影像数据范围 ACC F1-score 5×5 83.56% 83.52% 10×10 93.18% 93.31% 15×15 93.28% 93.39%
下载: 导出CSV
表 7 不同卷积层数的卷积核参数
Table 7. Convolution kernel parameters of different convolution layer numbers
模型名称 卷积层数 卷积核大小 每层卷积核个数 卷积神经网络 1 3×3 [256] 卷积神经网络 2 3×3 [256, 128] 卷积神经网络 3 3×3 [256, 128, 64]
下载: 导出CSV
表 8 各模型数据使用情况及实验结果
Table 8. Data usage of each model and experimental results
模型名称 地物地化数据 遥感影像数据 ACC F1-score 深度神经网络 √ × 82.13% 81.79% 卷积神经网络 × √ 93.42% 93.52% 多模态特征融合模型 √ √ 96.21% 96.21%
下载: 导出CSV
表 9 不同模型精确率
Table 9. Precision of different models
地质体 卷积神经网络 多模态特征融合模型 K1b流纹斑岩 84.45% 91.52% J3m沉凝灰岩 86.06% 90.27% P2z粉砂质板岩 86.35% 94.05% P3l中细粒长石岩屑砂岩 86.97% 91.38% J3m粗安岩 87.12% 93.39% K1b流纹质角砾晶屑玻屑凝灰岩 87.92% 95.43% δμ闪长玢岩 88.16% 93.09% K1m角闪辉石安山岩 88.77% 94.50% J3m流纹质含角砾岩屑晶屑熔结凝灰岩 88.83% 92.96% J3m流纹质沉角砾凝灰岩 94.99% 98.75%
下载: 导出CSV
-
Anderson, T. I., Vega, B., Kovscek, A. R., 2020. Multimodal Imaging and Machine Learning to Enhance Microscope Images of Shale. Computers & Geosciences, 145: 104593-104606. https://doi.org/10.1016/j.cageo.2020.104593 Arabi Aliabad, F., Shojaei, S., Zare, M., et al., 2019. Assessment of the Fuzzy ARTMAP Neural Network Method Performance in Geological Mapping Using Satellite Images and Boolean Logic. International Journal of Environmental Science and Technology, 16(7): 3829-3838. https://doi.org/10.1007/s13762-018-1795-7 Bhandari, D., Paul, S., Narayan, A., 2019. Multimodal Data Fusion and Prediction of Emotional Dimensions Using Deep Neural Network. Computational Intelligence: Theories, Applications and Future Directions, 799(2): 215-228. https://doi.org/10.1007/978-981-13-1135-2_17 Brandmeier, M., Chen, Y., 2019. Lithological Classification Using Multi-Sensor Data and Convolutional Neural Networks. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLⅡ-2/W16: 55-59. https://doi.org/10.5194/isprs-archives-xlii-2-w16-55-2019 Carter, A., Clift, P. D., 2008. Was the Indosinian Orogeny a Triassic Mountain Building or a Thermotectonic Reactivation Event? Comptes Rendus Geoscience, 340(2-3): 83-93. https://doi.org/10.1016/j.crte.2007.08.011 Cheng, G. J., Guo, W. H., 2017. Rock Images Classification by Using Deep Convolution Neural Network. Journal of Physics: Conference Series, 887(1): 012089. https://doi.org/10.1088/1742-6596/887/1/012089 Gao, S., 2020. A Review of Recent Researches and Reflections on Geospatial Artificial Intelligence. Geomatics and Information Science of Wuhan University, 45(12): 1865-1874(in Chinese with English abstract). He, J., Zhang, C.Q., Li, X.Z., et al., 2020. Survey of Research on Multimodal Fusion Technology for Deep Learning. Computer Engineering, 46(5): 1-11(in Chinese with English abstract). Krizhevsky, A., Hinton, G., 2009. Learning Multiple Layers of Features from Tiny Images. Handbook of Systemic Autoimmune Diseases, 1(4). Krizhevsky, A., Sutskever, I., Hinton, G. E., 2012. ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25(2): 1097-1105. https://doi.org/10.1145/3065386 LeCun, Y., Bottou, L., Bengio, Y., et al., 1998. Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11): 2278-2324. https://doi.org/10.1109/5.726791 Li, C. W., Zhang, R. S., Zhang, Z. T., et al., 2018. Tectonic Interpretation and Analysis Based on Multisource Remote Sensing Data: A Case Study of Jitai River in Chayi, Tibet. Remote Sensing Technology and Application, 33(4): 657-665(in Chinese with English abstract). Liu, J.W., Ding, X.H., Luo, X.L., 2020. Survey of Multimodal Deep Learning. Application Research of Computers, 37(6): 1601-1614(in Chinese with English abstract). Maimaitijiang, M., Sagan, V., Sidike, P., et al., 2019. Soybean Yield Prediction from UAV Using Multimodal Data Fusion and Deep Learning. Remote Sensing of Environment, 237: 111599. https://doi.org/10.1016/j.rse.2019.111599 Pires de Lima, R., Duarte, D., Nicholson, C., et al., 2020. Petrographic Microfacies Classification with Deep Convolutional Neural Networks. Computers & Geosciences, 142: 104481. https://doi.org/10.1016/j.cageo.2020.104481 Pour, A. B., Hashim, M., Makoundi, C., et al., 2016. Structural Mapping of the Bentong-Raub Suture Zone Using PALSAR Remote Sensing Data, Peninsular Malaysia: Implications for Sediment-Hosted/Orogenic Gold Mineral Systems Exploration. Resource Geology, 66(4): 368-385. https://doi.org/10.1111/rge.12105 Ran, X. J., Xue, L. F., Zhang, Y. Y., et al., 2019. Rock Classification from Field Image Patches Analyzed Using a Deep Convolutional Neural Network. Mathematics, 7(8): 755-770. https://doi.org/10.3390/math7080755 Russakovsky, O., Deng, J., Su, H., et al., 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3): 211-252. https://doi.org/10.1007/s11263-015-0816-y Simonyan, K., Zisserman, A., 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition". : arXiv: 1409.1556. https://arxiv.org/abs/1409.1556 Singer, D. A., 2021. How Deep Learning Networks could be Designed to Locate Mineral Deposits. Journal of Earth Science, 32(2): 288-292. https://doi.org/10.1007/s12583-020-1399-2 Su, Y. H., Fan, K., Bach, N., et al., 2019. Unsupervised Multi-Modal Neural Machine Translation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 10474-10483. https://doi.org/10.1109/CVPR.2019.01073 Tu, K., Wen, Q., Chen, H., et al., 2019. New Method of Structural Interpretation in Meadow Covering Based on GaoFen-3 Pol-SAR Images. Journal of Remote Sensing, 23(2): 243-251(in Chinese with English abstract). Wang, X., Huang, Q. Y., Celikyilmaz, A., et al., 2019. Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 6622-6631. https://doi.org/10.1109/CVPR.2019.00679 Xu, Y. Y., Li, Z. X., Xie, Z., et al., 2020. Prediction of Copper Mineralization Based on Semi-Supervised Neural Network. Earth Science, (12): 4563-4573(in Chinese with English abstract). Zhang, Y., Song, B., Du, X. J., et al., 2018. Vehicle Tracking Using Surveillance with Multimodal Data Fusion. IEEE Transactions on Intelligent Transportation Systems, 19(7): 2353-2361. https://doi.org/10.1109/TITS.2017.2787101 Zheng, M. Y., Tie, Y., Qi, L., et al., 2019. Dynamic Gesture Recognition Based on the Multimodality Fusion Temporal Segment Networks. 2019 8th International Symposium on Next Generation Electronics (ISNE). October 9-10, 2019, Zhengzhou, China. IEEE, 1-3. https://doi.org/10.1109/ISNE.2019.8896438 Zhou, T. X., Su, R., Guo, Y., et al., 2020. A Multi-Modality Fusion Network Based on Attention Mechanism for Brain Tumor Segmentation. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). April 3-7, 2020, Iowa City, IA, USA. IEEE, 377-380. https://doi.org/10.1109/ISBI45749.2020.9098392 Zuo, R.G., 2021. Data Science-Based Theory and Method of Quantitative Prediction of Mineral Resources. Earth Science Frontiers, 28(3): 49-55(in Chinese with English abstract). Zuo, R. G., Peng, Y., Li, T., et al., 2021. Challenges of Geological Prospecting Big Data Mining and Integration Using Deep Learning Algorithms. Earth Science, 46(1): 350-358(in Chinese with English abstract). 高松, 2020. 地理空间人工智能的近期研究总结与思考. 武汉大学学报(信息科学版), 45(12): 1865-1874. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH202012005.htm 何俊, 张彩庆, 李小珍, 等, 2020. 面向深度学习的多模态融合技术研究综述. 计算机工程, 46(5): 1-11. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC202005001.htm 李晨伟, 张瑞丝, 张竹桐, 等, 2018. 基于多源遥感数据的构造解译与分析: 以西藏察隅吉太曲流域为例. 遥感技术与应用, 33(4): 657-665. https://www.cnki.com.cn/Article/CJFDTOTAL-YGJS201804010.htm 刘建伟, 丁熙浩, 罗雄麟, 2020. 多模态深度学习综述. 计算机应用研究, 37(6): 1601-1614. https://www.cnki.com.cn/Article/CJFDTOTAL-JSYJ202006001.htm 涂宽, 文强, 谌华, 等, 2019. GF-3全极化影像在地表浅覆盖区进行地质构造解译的新方法. 遥感学报, 23(2): 243-251. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201902006.htm 徐永洋, 李孜轩, 谢忠, 等, 2020. 基于半监督神经网络的铜矿预测方法. 地球科学, 45(12): 4563-4573. doi: 10.3799/dqkx.2020.297 左仁广, 2021. 基于数据科学的矿产资源定量预测的理论与方法探索. 地学前缘, 28(3): 49-55. https://www.cnki.com.cn/Article/CJFDTOTAL-DXQY202103005.htm 左仁广, 彭勇, 李童, 等, 2021. 基于深度学习的地质找矿大数据挖掘与集成的挑战. 地球科学, 46(1): 350-358. doi: 10.3799/dqkx.2020.111 -
点击查看大图
计量
- 文章访问数: 432
- HTML全文浏览量: 617
- PDF下载量: 56
- 被引次数: 0
