Skip to main content

Research Repository

Advanced Search

Fusing deep learned and hand-crafted features of appearance, shape, and dynamics for automatic pain estimation

Egede, Joy Onyekachukwu; Valstar, Michel F.; Martinez, Brais

Fusing deep learned and hand-crafted features of appearance, shape, and dynamics for automatic pain estimation Thumbnail


Joy Onyekachukwu Egede

Michel F. Valstar

Brais Martinez


Automatic continuous time, continuous value assessment of a patient's pain from face video is highly sought after by the medical profession. Despite the recent advances in deep learning that attain impressive results in many domains, pain estimation risks not being able to benefit from this due to the difficulty in obtaining data sets of considerable size. In this work we propose a combination of hand-crafted and deep-learned features that makes the most of deep learning techniques in small sample settings. Encoding shape, appearance, and dynamics, our method significantly outperforms the current state of the art, attaining a RMSE error of less than 1 point on a 16-level pain scale, whilst simultaneously scoring a 67.3% Pearson correlation coefficient between our predicted pain level time series and the ground truth.

Conference Name 12th IEEE Conference on Face and Gesture Recognition (FG 2017)
End Date Jun 3, 2017
Acceptance Date Jan 23, 2017
Online Publication Date Jun 29, 2017
Publication Date May 30, 2017
Deposit Date Feb 27, 2017
Publicly Available Date May 30, 2017
Peer Reviewed Peer Reviewed
Keywords Pain, Estimation, Feature extraction, Face, Shape, Physiology, Machine learning
Public URL
Publisher URL
Related Public URLs
Additional Information Published in 2017 IEEE 12th International Conference on Automatic Face & Gesture Recognition (FG 2017). Piscataway, N.J. : IEEE, c2017. Electronic ISBN: 978-1-5090-4023-0. pp. 689-696, doi:10.1109/FG.2017.87 © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.


Downloadable Citations