Skip to main content

Research Repository

Advanced Search

Dynamic Facial Models for Video-Based Dimensional Affect Estimation

Song, Siyang; S�nchez-Lozano, Enrique; Kumar Tellamekala, Mani; Shen, Linlin; Johnston, Alan; Valstar, Michel

Dynamic Facial Models for Video-Based Dimensional Affect Estimation Thumbnail


Authors

Siyang Song

Enrique S�nchez-Lozano

Mani Kumar Tellamekala

Linlin Shen

Michel Valstar



Abstract

Dimensional affect estimation from a face video is a challenging task, mainly due to the large number of possible facial displays made up of a set of behaviour primitives including facial muscle actions. The displays vary not only in composition but also in temporal evolution, with each display composed of behaviour primitives with varying in their short and long-term characteristics. Most existing work models affect relies on complex hierarchical recurrent models unable to capture short-term dynamics well. In this paper , we propose to encode these short-term facial shape and appearance dynamics in an image, where only the semantic meaningful information is encoded into the dynamic face images. We also propose binary dynamic facial masks to remove 'stable pixels' from the dynamic images. This process allows filtering of non-dynamic information, i.e. only pix-els that have changed in the sequence are retained. Then, the final proposed Dynamic Facial Model (DFM) encodes both filtered facial appearance and shape dynamics of a image sequence preceding to the given frame into a three-channel raster image. A CNN-RNN architecture is tasked with modelling primarily the long-term changes. Experiments show that our dynamic face images achieved superior performance over the standard RGB face images on dimensional affect prediction task.

Citation

Song, S., Sánchez-Lozano, E., Kumar Tellamekala, M., Shen, L., Johnston, A., & Valstar, M. (2019). Dynamic Facial Models for Video-Based Dimensional Affect Estimation. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) (1608-1617). https://doi.org/10.1109/ICCVW.2019.00200

Conference Name 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Conference Location Seoul, Korea (South)
Start Date Oct 27, 2019
End Date Oct 28, 2019
Acceptance Date Aug 20, 2019
Online Publication Date Mar 5, 2020
Publication Date 2019-10
Deposit Date Nov 1, 2019
Publicly Available Date Nov 1, 2019
Publisher Institute of Electrical and Electronics Engineers
Pages 1608-1617
Series ISSN 2473-9944
Book Title Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
ISBN 978-1-7281-5024-6
DOI https://doi.org/10.1109/ICCVW.2019.00200
Public URL https://nottingham-repository.worktribe.com/output/3010204
Publisher URL https://ieeexplore.ieee.org/document/9022266
Related Public URLs http://openaccess.thecvf.com/content_ICCVW_2019/html/CVPM/Song_Dynamic_Facial_Models_for_Video-Based_Dimensional_Affect_Estimation_ICCVW_2019_paper.html
Additional Information © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Files





You might also like



Downloadable Citations