Abstract:

Aiming at the problem of insufficient representation of interactive semantic information in the double human interaction behavior recognition method based on graph convolutional neural networks，a new double human interactive spatial?temporal graph convolution network (DHI?STGCN) was proposed for behavior recognition. The network contains spatial sub?network modules and temporal sub?network modules. Based on the 3D skeleton data obtained from the interactive action video, a spatial action graph of double human interactive action was generated for the representation of spatial information. In the graph, the connecting edges between double human were given different weights according to the joint point position information. The connection of context time information was added in the constructed adjacency matrix, and the joint points in the graph were connected with their nodes within a certain time range in time information processing. The generated spatial?temporal graph data was sent to the spatial graph convolution network module, and the temporal graph convolution network module was combined to enhance the continuity of inter frame motion features for modeling in time. The model fully considers the close relationship of double human interaction. The comparative experimental results on NTU?RGB+D dataset show that the algorithm has strong robustness and obtains better interaction recognition effect than the existing models.

Key words: Spatial?temporal graph convolution, Skeleton data, Double human interaction, Behavior recognition

摘要：

针对图卷积神经网络的双人交互行为识别方法存在交互语义信息表达不充分的问题，提出了一种新的双人交互时空图卷积神经网络（DHI?STGCN）用于行为识别的方法。该网络包含空间子网络模块和时间子网络模块。将基于交互动作视频获取的3D骨架数据生成一种双人交互动作的空间动作图用于空间信息的表示，图中根据关节点位置信息对双人之间的连接边赋予不同的权重。时间信息处理中，在构造的邻接矩阵中增加了上下文时间信息的联系，图中关节点与其一定时间范围内的节点增加连接。将生成的时空图数据送入空间图卷积网络模块，结合时间图卷积网络模块增强帧间运动特征连续性进行时序建模。该模型充分考虑了双人交互动作的紧密关系，具有较强的鲁棒性，获得了比现有模型更好的交互动作识别效果。

关键词: 时空图卷积, 骨架数据, 双人交互, 行为识别