Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features

Song, Siyang; Shen, Linlin; Valstar, Michel F.

Abstract

Depression is a serious mental disorder that affects millions of people all over the world. Traditional clinical diagnosis methods are subjective, complicated and need extensive participation of experts. Audio-visual automatic depression analysis systems predominantly base their predictions on very brief sequential segments, sometimes as little as one frame. Such data contains much redundant information, causes a high computational load, and negatively affects the detection accuracy. Final decision making at the sequence level is then based on the fusion of frame or segment level predictions. However, this approach loses longer term behavioural correlations, as the behaviours themselves are abstracted away by the frame-level predictions. We propose to on the one hand use automatically detected human behaviour primitives such as Gaze directions, Facial action units (AU), etc. as low-dimensional multi-channel time series data, which can then be used to create two sequence descriptors. The first calculates the sequence-level statistics of the behaviour primitives and the second casts the problem as a Convolutional Neural Network problem operating on a spectral representation of the multichannel behaviour signals. The results of depression detection (binary classification) and severity estimation (regression) experiments conducted on the AVEC 2016 DAIC-WOZ database show that both methods achieved significant improvement compared to the previous state of the art in terms of the depression severity estimation.

Citation

Song, S., Shen, L., & Valstar, M. F. (2018). Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features. In 13th IEEE International Conference on Automatic Face and Gesture Recognition: FG2018: 15-19 May 2018 Xi'an, China (158-166). https://doi.org/10.1109/FG.2018.00032

• Conference Name: 13th IEEE International Conference on Face and Gesture Recognition (FG 2018)
• Start Date: May 15, 2018
• End Date: May 19, 2018
• Acceptance Date: Jan 25, 2018
• Online Publication Date: Jun 7, 2018
• Publication Date: May 17, 2018
• Deposit Date: Apr 30, 2018
• Publicly Available Date: May 17, 2018
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Pages: 158-166
• Book Title: 13th IEEE International Conference on Automatic Face and Gesture Recognition: FG2018: 15-19 May 2018 Xi'an, China
• ISBN: 9781538623350
• DOI: https://doi.org/10.1109/FG.2018.00032
• Public URL: https://nottingham-repository.worktribe.com/output/933157
• Publisher URL: https://ieeexplore.ieee.org/document/8373825/
• Related Public URLs: https://fg2018.cse.sc.edu/
• Additional Information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.