A dynamic appearance descriptor approach to facial actions temporal modeling

Jiang, Bihan; Valstar, Michel; Martinez, Brais; Pantic, Maja

Abstract

Both the configuration and the dynamics of facial expressions are crucial for the interpretation of human facial behavior. Yet to date, the vast majority of reported efforts in the field either do not take the dynamics of facial expressions into account, or focus only on prototypic facial expressions of six basic emotions. Facial dynamics can be explicitly analyzed by detecting the constituent temporal segments in Facial Action Coding System (FACS) Action Units (AUs)-onset, apex, and offset. In this paper, we present a novel approach to explicit analysis of temporal dynamics of facial actions using the dynamic appearance descriptor Local Phase Quantization from Three Orthogonal Planes (LPQ-TOP). Temporal segments are detected by combining a discriminative classifier for detecting the temporal segments on a frame-by-frame basis with Markov Models that enforce temporal consistency over the whole episode. The system is evaluated in detail over the MMI facial expression database, the UNBC-McMaster pain database, the SAL database, the GEMEP-FERA dataset in database-dependent experiments, in cross-database experiments using the Cohn-Kanade, and the SEMAINE databases. The comparison with other state-of-the-art methods shows that the proposed LPQ-TOP method outperforms the other approaches for the problem of AU temporal segment detection, and that overall AU activation detection benefits from dynamic appearance information.

Citation

Jiang, B., Valstar, M., Martinez, B., & Pantic, M. (2014). A dynamic appearance descriptor approach to facial actions temporal modeling. IEEE Transactions on Cybernetics, 44(2), https://doi.org/10.1109/TCYB.2013.2249063

• Journal Article Type: Article
• Acceptance Date: Jan 14, 2014
• Publication Date: Feb 1, 2014
• Deposit Date: Oct 7, 2016
• Publicly Available Date: Oct 7, 2016
• Journal: IEEE Transactions on Cybernetics
• Electronic ISSN: 2168-2275
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 44
• Issue: 2
• DOI: https://doi.org/10.1109/TCYB.2013.2249063
• Keywords: Facial dynamics, action unit detection, dynamic appearance descriptors, LPQ-TOP, temporal segment detection
• Public URL: https://nottingham-repository.worktribe.com/output/720684
• Publisher URL: http://ieeexplore.ieee.org/document/6504484/
• Additional Information: © 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.