Fusing deep learned and hand-crafted features of appearance, shape, and dynamics for automatic pain estimation

Egede, Joy Onyekachukwu; Valstar, Michel F.; Martinez, Brais

Abstract

Automatic continuous time, continuous value assessment of a patient's pain from face video is highly sought after by the medical profession. Despite the recent advances in deep learning that attain impressive results in many domains, pain estimation risks not being able to benefit from this due to the difficulty in obtaining data sets of considerable size. In this work we propose a combination of hand-crafted and deep-learned features that makes the most of deep learning techniques in small sample settings. Encoding shape, appearance, and dynamics, our method significantly outperforms the current state of the art, attaining a RMSE error of less than 1 point on a 16-level pain scale, whilst simultaneously scoring a 67.3% Pearson correlation coefficient between our predicted pain level time series and the ground truth.

Citation

Egede, J. O., Valstar, M. F., & Martinez, B. (2017). Fusing deep learned and hand-crafted features of appearance, shape, and dynamics for automatic pain estimation.

• Conference Name: 12th IEEE Conference on Face and Gesture Recognition (FG 2017)
• End Date: Jun 3, 2017
• Acceptance Date: Jan 23, 2017
• Online Publication Date: Jun 29, 2017
• Publication Date: May 30, 2017
• Deposit Date: Feb 27, 2017
• Publicly Available Date: May 30, 2017
• Peer Reviewed: Peer Reviewed
• Keywords: Pain, Estimation, Feature extraction, Face, Shape, Physiology, Machine learning
• Public URL: https://nottingham-repository.worktribe.com/output/862392
• Publisher URL: http://ieeexplore.ieee.org/abstract/document/7961808/
• Related Public URLs: http://www.fg2017.org/
• Additional Information: Published in 2017 IEEE 12th International Conference on Automatic Face & Gesture Recognition (FG 2017). Piscataway, N.J. : IEEE, c2017. Electronic ISBN: 978-1-5090-4023-0. pp. 689-696, doi:10.1109/FG.2017.87 © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
• Contract Date: Feb 27, 2017