Skip to main content

Research Repository

See what's under the surface

A dynamic appearance descriptor approach to facial actions temporal modeling

Jiang, Bihan; Valstar, Michel; Martinez, Brais; Pantic, Maja

Authors

Bihan Jiang

Michel Valstar

Brais Martinez

Maja Pantic



Abstract

Both the configuration and the dynamics of facial expressions are crucial for the interpretation of human facial behavior. Yet to date, the vast majority of reported efforts in the field either do not take the dynamics of facial expressions into account, or focus only on prototypic facial expressions of six basic emotions. Facial dynamics can be explicitly analyzed by detecting the constituent temporal segments in Facial Action Coding System (FACS) Action Units (AUs)-onset, apex, and offset. In this paper, we present a novel approach to explicit analysis of temporal dynamics of facial actions using the dynamic appearance descriptor Local Phase Quantization from Three Orthogonal Planes (LPQ-TOP). Temporal segments are detected by combining a discriminative classifier for detecting the temporal segments on a frame-by-frame basis with Markov Models that enforce temporal consistency over the whole episode. The system is evaluated in detail over the MMI facial expression database, the UNBC-McMaster pain database, the SAL database, the GEMEP-FERA dataset in database-dependent experiments, in cross-database experiments using the Cohn-Kanade, and the SEMAINE databases. The comparison with other state-of-the-art methods shows that the proposed LPQ-TOP method outperforms the other approaches for the problem of AU temporal segment detection, and that overall AU activation detection benefits from dynamic appearance information.

Journal Article Type Article
Publication Date Feb 1, 2014
Journal IEEE Transactions on Cybernetics
Electronic ISSN 2168-2275
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 44
Issue 2
Institution Citation Jiang, B., Valstar, M., Martinez, B., & Pantic, M. (2014). A dynamic appearance descriptor approach to facial actions temporal modeling. IEEE Transactions on Cybernetics, 44(2), doi:10.1109/TCYB.2013.2249063
DOI https://doi.org/10.1109/TCYB.2013.2249063
Keywords Facial dynamics, action unit detection, dynamic appearance descriptors, LPQ-TOP, temporal segment detection
Publisher URL http://ieeexplore.ieee.org/document/6504484/
Copyright Statement Copyright information regarding this work can be found at the following address: http://eprints.nottingh.../end_user_agreement.pdf
Additional Information © 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.

Files

TAUD_preprint.pdf (3.2 Mb)
PDF

Copyright Statement
Copyright information regarding this work can be found at the following address: http://eprints.nottingham.ac.uk/end_user_agreement.pdf





Downloadable Citations