Abstract: In the field of human interaction recognition, local features based on RGB video often cannot effectively distinguish approximate actions. The Depth image information and the color image information are merged in the recognition process, and a two⁃person interactive behavior recognition algorithm that integrates the depth information and the individual segmentation fusion is proposed.The algorithm firstly extracts the points of interest for RGB and Depth video, then uses 3DSIFT to describe the features on RGB video. The YOLO network is introduced into divide the left and right points of interest on the Depth video, and the local co⁃occurrence matrix is used for local correlation information description. Finally, the nearest neighbor classifier is used to classify the RGB features and Depth features, and further the recognition results are obtained by the decision⁃level fusion, which improves the accuracy of recognition. The results show that the combination of depth visual co⁃occurrence matrix can greatly improve the recognition accuracy of double interaction behavior, and the correct recognition rate of 90% of the actions in SBU Kinect interaction database can verify the effectiveness of the proposed algorithm.

Key words: Interaction behavior, Local features, Co?occurrence visual matrix, Depth features, YOLO, Decision level fusion

摘要： 在人类交互行为识别领域，基于RGB视频的局部特征往往不能有效区分近似动作，将深度图像（Depth）与彩色图像（RGB）在识别过程中进行融合，提出一种融合Depth信息的整体和个体分割融合的双人交互行为识别算法。该算法首先分别对RGB和Depth视频进行兴趣点提取，在RGB视频上采用3DSIFT进行特征描述，在Depth视频上利用YOLO网络对左右两人兴趣点进行划分，并使用视觉共生矩阵对局部关联信息进行描述。最后使用最近邻分类器分别对RGB特征和Depth特征进行分类识别，进一步通过决策级融合两者识别结果，提高识别准确率。结果表明，结合深度视觉共生矩阵可以大大提高双人交互行为识别准确率，对于SBU Kinect interaction数据库中的动作可以达90%的正确识别率，验证了所提算法的有效性。

关键词: 交互行为, 局部特征, 视觉共生矩阵, 深度特征, YOLO, 决策级融合