提出了一种基于卷积神经网络和长短期记忆(LSTM)神经网络的深度学习网络结构。采用特征融合的方法,通过卷积网络提取出浅层特征与深层特征并进行联接,对特征通过卷积进行融合,将获得的矢量信息输入LSTM单元。分别使用数据光流信息与红绿蓝信息训练网络,将各网络的结果进行加权融合。实验结果表明,所提模型有效地提高了行为识别精度。

A deep learning network structure based on the convolutional neural network and long short term memory (LSTM) neural network is proposed. The feature fusion is used to extract the shallow features and deep features through the convolutional network, and the features are fused by convolution, and the the obtained vector information is input into the LSTM unit. Networks are trained separately using the optical flow images and the red green blue information, and the results from each network are fused with weights. The experimental results show that the proposed model effectively improves the accuracy of behavior recognition.