Smoking behavior detection in public places has become the focus of social attention. At present, smoking behavior detection based on machine vision mainly uses deep learning to detect cigarette ends and decide whether there is a smoking behavior by whether there are cigarette ends. Because the cigarette end target is small, the accuracy of this method is relatively low.Through a large number of observations, it is found that smoking behavior is rhythmic and cyclical. Therefore,a smoking behavior detection method based on human keypoints and YOLOv4 is proposed.On the basis of deep learning to detect cigarette butts, the keypoints detection of human body is added to judge whether smoking action occurs by calculating distance, angle and time cycle. Firstly, AlphaPose and RetinaFace are used to obtain the position information of key points of human body.Secondly, based on the position information of keypoints, the distance between hand and mouth, the angle between hand, elbow and shoulder, and the time period of smoking are calculated, and the smoking behavior rules are set.Finally, the cigarette end in the image is detected with YOLOv4 to determine whether there is smoking behavior. The experimental results show that the smoking behavior is detected timely and effectively in the self collected smoking data.
[1] 陈睿龙,罗磊,蔡志平,等.基于深度学习的实时吸烟检测算法[J].计算机科学与探索,2021,15(2):327-337.
CHEN R L,LUO L,CAI Z P,et al.Algorithm for real-time smoking detection based on deep learning[J].Journal of Frontiers of Computer Science and Technology,2021,15(2):327-337.
[2] 汪祖云,廖惠敏,张日东,等.结合烟雾多特征的出租车司机吸烟行为检测[C]//第十二届全国信号和智能信息处理与应用学术会议论文集. 北京:中国高科技产业化研究会,2018:368-373.
WANG Z Y, LIAO H M, ZHANG R D, et al. Detection of smoking behavior of taxi drivers based on smoke characteristics[C]//The 12th National Conference on Signal and Intelligent Information Processing and Application. Beijing:China High-Tech Industrialization Association, 2018: 368-373.
[3] 吴帅,徐勇,赵东宁.基于深度卷积网络的目标检测综述[J].模式识别与人工智能,2018,31(4):335-346.
WU S,XU Y,ZHAO D N.Survey of object detection based on deep convolutional network[J].Pattern Recognition and Artificial Intelligence,2018,31(4):335-346.
[4] 欧先锋,晏鹏程,王汉谱,等.基于深度帧差卷积神经网络的运动目标检测方法研究[J].电子学报,2020,48(12):2384-2393.
OU X F,YAN P C,WANG H P,et al.Research of moving object detection based on deep frame difference convolution neural network[J].Acta Electronica Sinica,2020,48(12):2384-2393.
[5] 张顺,龚怡宏,王进军.深度卷积神经网络的发展及其在计算机视觉领域的应用[J].计算机学报,2019,42(3):453-482.
ZHANG S,GONG Y H,WANG J J.The development of deep convolution neural network and its applications on computer vision[J].Chinese Journal of Computers,2019,42(3):453-482.
[6] 刘婧,杨旭,刘董经典,等.基于人体关节点的多人吸烟动作识别算法[J].计算机工程与应用,2021,57(1):234-241.
LIU J,YANG X,LIU D J D,et al.Multi-person smoking action recognition algorithm based on human joint points[J].Computer Engineering and Applications,2021,57(1):234-241.
[7] 李倩.基于深度学习的烟支检测技术研究与应用[D].西安:西安邮电大学,2020.
LI Q.Research and application of cigarette detection technology based on deep learning[D].Xi'an:Xi'an University of Posts and Telecommunications,2020.
[8] 韩贵金,李倩.基于Faster R-CNN的吸烟快速检测算法[J].西安邮电大学学报,2020,25(2):85-91.
HAN G J,LI Q.A rapid detection algorithm for smoking based on Faster R-CNN[J].Journal of Xi'an University of Posts and Telecommunications,2020,25(2):85-91.
[9] 马晓菲.基于人脸分析的公共场所吸烟行为检测系统研究[D].秦皇岛:燕山大学,2020.
MA X F.Research on smoking behavior detection systems in public places based on face analysis[D].Qinhuangdao:Yanshan University,2020.
[10] 邓益侬,罗健欣,金凤林.基于深度学习的人体姿态估计方法综述[J].计算机工程与应用,2019,55(19):22-42.
DENG Y N,LUO J X,JIN F L.Overview of human pose estimation methods based on deep learning[J].Computer Engineering and Applications,2019,55(19):22-42.
[11] 孟琭,高恒上,张含光,等.基于全连接神经网络的三维人体姿态估计[J].仪器仪表学报,2020,41(10):165-177.
MENG L,GAO H S,ZHANG H G,et al.Three-dimensional human pose estimation based on the fully connected neural network[J].Chinese Journal of Scientific Instrument,2020,41(10):165-177.
[12] 刘一敏,蒋建国,齐美彬,等.融合生成对抗网络和姿态估计的视频行人再识别方法[J].自动化学报,2020,46(3):576-584.
LIU Y M,JIANG J G,QI M B,et al.Video-based person re-identification method based on GAN and pose estimation[J].Acta Automatica Sinica,2020,46(3):576-584.
[13] 蔡轶珩,王雪艳,马杰,等.基于随机森林误分类处理的3D人体姿态估计[J].自动化学报,2020,46(7):1457-1466.
CAI Y H,WANG X Y,MA J,et al.3D human pose estimation based on random forest misclassification processing mechanism[J].Acta Automatica Sinica,2020,46(7):1457-1466.
[14] 唐心宇,宋爱国.人体姿态估计及在康复训练情景交互中的应用[J].仪器仪表学报,2018,39(11):195-203.
TANG X Y,SONG A G.Human pose estimation and its implementation in scenario interaction system of rehabilitation training[J].Chinese Journal of Scientific Instrument,2018,39(11):195-203.
[15] 杨观赐,杨静,苏志东,等.改进的YOLO特征提取算法及其在服务机器人隐私情境检测中的应用[J].自动化学报,2018,44(12):2238-2249.
YANG G C,YANG J,SU Z D,et al.An improved YOLO feature extraction algorithm and its application to privacy situation detection of social robots[J].Acta Automatica Sinica,2018,44(12):2238-2249.
[16] GHULI P,SHASHANK B N,RAO A G.Development of framework for detecting smoking scene in video clips[J].Indonesian Journal of Electrical Engineering and Computer Science,2019,13(1):22.
[17] VAROL G,LAPTEV I,SCHMID C.Long-term temporal convolutions for action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(6):1510-1517.
