Efficient hierarchical temporal segmentation method for facial expression sequences

Abstract

Temporal segmentation of facial expression sequences is important to understand and analyze human facialexpressions. It is, however, challenging to deal with the complexity of facial muscle movements by finding a suitablemetric to distinguish among different expressions and to deal with the uncontrolled environmental factors in the realworld. This paper presents a two-step unsupervised segmentation method composed of rough segmentation and finesegmentation stages to compute the optimal segmentation positions in video sequences to facilitate the segmentation ofdifferent facial expressions. The proposed method performs localization of facial expression patches to aid in recognitionand extraction of specific features. In the rough segmentation stage, facial sequences are segmented into distinct facialbehaviors based on the similarity between sequence frames, while similarity between segments is computed to obtainoptimal segmentation positions in the fine segmentation stage. The proposed method has been evaluated in experimentsusing the MMI dataset and real videos. Experiment results compared to other state-of-the-art methods indicate betterperformance of the proposed method.