基于图像分块和多头注意力机制的气象识别研究

doi:10.12422/j.issn.1672-6952.2024.02.013

辽宁石油化工大学学报 ›› 2024, Vol. 44 ›› Issue (2): 83-90.DOI: 10.12422/j.issn.1672-6952.2024.02.013

基于图像分块和多头注意力机制的气象识别研究

赵旭峰¹(), 刘琳琳¹, 曹宇¹(), 叶成荫¹, 郭宗凯²

^1.辽宁石油化工大学信息与控制工程学院，辽宁抚顺 113001
^2.辽宁省气象装备保障中心，辽宁沈阳 110166

收稿日期:2022-08-10 修回日期:2022-09-02 出版日期:2024-04-25 发布日期:2024-04-24
通讯作者: 曹宇
作者简介:赵旭峰（1996⁃），男，硕士研究生，从事机器学习、深度学习算法的研究；E⁃mail：925874520@qq.com。
基金资助:
辽宁省重点研发计划项目(2020JH2/10300040);辽宁省教育厅科学研究项目(L2020031);辽宁省应用基础研究计划项目(2022JH2/101300272)

Research on Weather Recognition Based on Image Segmentation and Multi⁃Head Attention Mechanism

Xufeng ZHAO¹(), Linlin LIU¹, Yu CAO¹(), Chengyin YE¹, Zongkai GUO²

^1.School of Information and Control Engineering，Liaoning Petrochemical University，Fushun Liaoning 113001，China
^2.Liaoning Meteorological Equipment Support Center，Shenyang Liaoning 110166，China

Received:2022-08-10 Revised:2022-09-02 Published:2024-04-25 Online:2024-04-24
Contact: Yu CAO

摘要/Abstract

摘要：

基于图像对天气现象进行识别，对天气状况的分析至关重要。针对传统的机器学习方法对各类天气特征难以准确提取且天气现象分类效果差，以及深度学习对天气现象识别的准确率不高的问题，提出了基于图像分块和多头注意力机制的天气识别模型。该模型首次将Swin Transformer引入天气识别领域中，采用了窗口多头自注意层与移位窗口多头自注意层相结合的多头注意力机制。结果表明，其区域相关特征提取能力弥补传统方法的不足，能够提取图像中复杂的天气特征。采用迁移学习对模型进行训练，将微调模型的全连接参数输入到Softmax分类器，实现了对多类别天气图像的识别，识别准确率为99.20%，优于对比的几种主流方法。因此，该方法可以作为天气识别模块应用于地面气象识别系统。

关键词: 天气识别, 图像分块, 多头注意力机制, 区域相关特征, 迁移学习

Abstract:

Recognition of weather phenomena based on images is essential for the analysis of weather conditions. To address the problems that traditional machine learning methods are difficult to accurately extract various weather features and poor in classifying weather phenomena and the accuracy of deep learning for weather phenomena recognition is not high, a weather recognition model based on image block and multi?headed attention mechanism is proposed. The model introduces Swin Transformer into the field of weather recognition for the first time, and adopts a multi?headed attention mechanism combining window multi?head self?attention layer and shifted?window multi?head self?attention layer, whose regionally relevant features extraction capability makes up for the shortcomings of traditional methods and can extract complex weather features from images. The model is trained using transfer learning, and the fully connected parameters of the fine?tuned model are input to the Softmax classifier to achieve recognition of multi?category weather images with 99.20% recognition accuracy, which is better than several mainstream methods in comparison, and it can be applied to ground weather recognition systems as a weather recognition module.

Key words: Weather recognition, Image block, Multi?headed attention mechanism, Regional correlation features, Transfer learning

中图分类号:

TP391.4

赵旭峰, 刘琳琳, 曹宇, 叶成荫, 郭宗凯. 基于图像分块和多头注意力机制的气象识别研究[J]. 辽宁石油化工大学学报, 2024, 44(2): 83-90.

Xufeng ZHAO, Linlin LIU, Yu CAO, Chengyin YE, Zongkai GUO. Research on Weather Recognition Based on Image Segmentation and Multi⁃Head Attention Mechanism[J]. Journal of Liaoning Petrochemical University, 2024, 44(2): 83-90.

图/表 10

图1 AlexNet网络架构

Fig.1 AlexNet network architecture

图2 Inception模块

Fig.2 Inception module

图3 残差单元（a）浅层网络（b）深层网络

Fig.3 Residual unit

图4 基于图像分块和多头注意力机制的气象识别模型

Fig.4 Weather recognition model based on image segmentation and multi?head attention mechanism

图5 ViT?B/16架构

Fig.5 ViT?B/16 architecture

图6 Swin?T架构

Fig.6 Swin?T architecture

表1 气象识别任务中各模型的参数设置

Table 1 Parameter setting of each model in meteorological identification task

模型	块大小	学习率	优化器
AlexNet	32	0.000 2	Adam
VGG16/19	16	0.000 1	Adam
GoogLeNet	32	0.000 3	Adam
ResNet34/50/101	16	0.000 1	Adam
ViT⁃B/L⁃16/32	8	0.001 0	SGD
Swin⁃T/S/B/L	8	0.000 1	AdamW

图7 不同模型的验证准确率

Fig.7 Validation accuracy of different models

表2 各模型识别准确率

Table 2 Recognition accuracy of each model

模型	识别准确率/%
Swin⁃L	99.20
Swin⁃B	98.94
ResNet50	97.15
ViT⁃L/16	97.14
ResNet101	97.10
Swin⁃S	97.09
Swin⁃T	96.93
ResNet34	96.89
VGG16	96.44
ViT⁃B/32	96.08
VGG19	95.73
GoogLeNet	95.68
ViT⁃L/32	95.53
ViT⁃B/16	95.48
AlexNet	90.67

表3 Swin?L模型在测试集上的评价结果

Table 3 Evaluation result of Swin?L model on the test set

天气	准确率/%	召回率/%	F_l 测量值
雾天	99.40	98.80	0.991 0
雨天	97.62	97.62	0.976 2
雪天	98.57	98.57	0.985 7
晴天	99.53	99.81	0.996 7

参考文献 17

1	CAI Z， JIANG F， CHEN J M， et al. Weather condition dominates regional PM2.5 pollutions in the eastern coastal provinces of China during winter［J］. Aerosol and Air Quality Research， 2018， 18（4）： 969⁃980.
2	李骞，范茵，张璟，等. 基于室外图像的天气现象识别方法［J］. 计算机应用， 2011， 31（6）： 1624⁃1627.
	LI Q， FAN Y， ZHANG J， et al. Method of weather recognition based on decision⁃tree⁃based SVM［J］. Journal of Computer Applications， 2011， 31（6）： 1624⁃1627.
3	ZHANG Z， MA H D， FU H Y， et al. Scene⁃free multi⁃class weather classification on single images［J］. Neurocomputing， 2016， 207： 365⁃373.
4	LIN D， LU C W， HUANG H， et al. RSCM： Region selection and concurrency model for multi⁃class weather recognition［J］. IEEE Transactions on Image Processing： A Publication of the IEEE Signal Processing Society， 2017， 26（9）： 4154⁃4167.
5	TAHERKHANI A， COSMA G， MCGINNITY T M. AdaBoost⁃CNN： An adaptive boosting algorithm for convolutional neural networks to classify multi⁃class imbalanced datasets using transfer learning［J］. Neurocomputing， 2020，404： 351⁃366.
6	XIAO H X， ZHANG F， SHEN Z P， et al. Classification of weather phenomenon from images by using deep convolutional neural network［J］. Earth and Space Science， 2021， 8（5）： e2020EA001604.
7	郭志强，胡永武，刘鹏，等. 基于特征融合的室外天气图像分类［J］. 计算机应用， 2020， 40（4）： 1023⁃1029.
	GUO Z Q， HU Y W， LIU P， et al. Outdoor weather image classification based on feature fusion［J］. Journal of Computer Applications， 2020， 40（4）： 1023⁃1029.
8	左杰格，柳晓鸣，蔡兵. 基于图像分块与特征融合的户外图像天气识别［J］. 计算机科学， 2022， 49（3）： 197⁃203.
	ZUO J G， LIU X M， CAI B. Outdoor image weather recognition based on image blocks and feature fusion［J］. Computer Science， 2022， 49（3）： 197⁃203.
9	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York：Curran Associates Inc， 2017： 6000⁃6010.
10	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84⁃90.
11	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large⁃scale image recognition［EB/OL］.（2014⁃09⁃04）［2022⁃07⁃25］. https：//arxiv.org/abs/1409.1556.
12	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］//2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. New York： Institute of Electrical and Electronics Engineers， 2015： 1⁃9.
13	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［EB/OL］. （2015⁃12⁃10）［2022⁃07⁃25］. https：//arxiv.org/abs/1512.03385.
14	DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16×16 words： Transformers for image recognition at scale［EB/OL］.（2020⁃10⁃22）［2022⁃07⁃25］. https：//arxiv.org/abs/2010.11929.
15	曹英丽，方诗琦，王洋，等.分布式光伏发电特性与气象影响因子诊断分析［J］.沈阳农业大学学报，2018，49（3）:363⁃370.
	CAO Y L， FANG S Q， WANG Y， et al. Diagnostic analysis of distributed photovoltaic power characteristics and the impact of meteorological factors［J］.Journal of Shenyang Agricultural University，2018，49（3）：363⁃370.
16	景源，宫玉莹.基于深度学习的单幅图像超分辨率重建方法研究［J］.辽宁大学学报，2022，49（3）：225⁃231.
	JING Y， GONG Y Y. Research on super⁃resolution reconstruction of single image based on depth learning［J］.Journal of Liaoning University，2022，49（3）：225⁃231.
17	LIU Z， LIN Y T， CAO Y， et al. Swin transformer： Hierarchical vision transformer using shifted windows［C］//2021 IEEE/CVF International Conference on Computer Vision （ICCV）. New York： Institute of Electrical and Electronics Engineers， 2021： 9992⁃10002.

[1]	王国良, 任允帅, 王阳. 一种基于马尔科夫随机场的脑MR图像分割改进算法[J]. 辽宁石油化工大学学报, 2023, 43(1): 89-96.
[2]	汤永恒, 潘斌. 基于双向转换网络的域自适应单幅图像去雾方法[J]. 辽宁石油化工大学学报, 2022, 42(6): 78-83.
[3]	李文娜. 基于小波变换⁃海森伯格矩阵⁃奇异值分解的图像水印算法[J]. 辽宁石油化工大学学报, 2022, 42(5): 84-89.

基于图像分块和多头注意力机制的气象识别研究

Research on Weather Recognition Based on Image Segmentation and Multi⁃Head Attention Mechanism

HTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 17

相关文章 3

编辑推荐

Metrics