为有效利用行为视频的长时时域信息,提高行为识别准确率,提出一种结合有序光流图和双流卷积神经网络的行为识别算法。首先利用Rank支持向量机(SVM)算法将连续光流序列压缩总结成单幅有序光流图,实现对视频长时时域结构的建模;然后设计一个包含表观和短时运动流与长时运动流的双流卷积网络,分别以堆叠RGB帧、有序光流图为输入提取视频的表观和短时运动信息与长时运动信息;最后将双流网络的C3D描述子和VGG描述子融合后输入线性SVM进行行为识别。在HMDB51和UCF101两个数据集的实验结果表明,该算法能够有效利用空域表观信息和时域运动信息,具有较高的行为视频识别准确率。

In order to effectively utilize the long-term temporal information of video for improving the accuracy of action recognition, a new recognition approach is proposed based on the sequential optical flow image and double-stream convolutional neural networks. Firstly, the Rank support vector machine (SVM) algorithm is used to compress the continuous optical flow frames into a single sequential optical flow image to realize the modeling of the long-term temporal structure of video. Secondly, we design a double-stream convolutional networks containing appearance and short-term motion stream and long-term motion stream. It takes the stacked RGB frames and the sequential optical flow images as input to extract the appearance and short-time motion information and the long-time motion information of the video. Finally, the linear SVM is adopted to integrate C3D descriptor and VGG descriptor for action recognition. The experimental results on HMDB51 and UCF101 datasets show that the proposed approach improves the action recognition accuracy effectively by using the spatial information and the temporal motion information.