Human activity recognition by using MHIs of frame sequences

Abstract

A motion history image (MHI) is a temporal template that collapses temporal motion information into asingle image in which intensity is a function of recency of motion. In recent years, the popularity of deep learningarchitectures for human activity recognition has encouraged us to explore the effectiveness of combining them andMHIs. Based on this, two new methods are introduced in this paper. In the first method, which is called the basicmethod, each video splits into N groups of consecutive frames, and the MHI is calculated for each group. Transferlearning with the fine-tuning technique is used for classifying these temporal templates. The experimental results showthat some misclassification errors are created because of the similarities between these temporal templates; these errorscan be corrected by detecting specific objects in the scenes. Thus, spatial information consisting of a single frame isalso added to the second method, called the proposed method. By converting video classification problems into imageclassification problems in the proposed method, less memory is needed and the time complexity is greatly reduced. Theyare implemented and compared with state-of-the-art approaches on two data sets. The results show that the proposedmethod significantly outperforms the others. It achieves recognition accuracies of 92% and 92.4% for the UCF Sportand UCF-11 action data sets, respectively.