Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms
Published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
Abstract: Different from the conventional facial expression, micro-expression is an involuntary and transient facial expression, which can reveal a genuine emotion that people attempt to hide. The detection and recognition of micro-expressions are difficult and heavily rely on expert experiences, since micro-expressions are transient and of low intensity. Due to its intrinsic particularity and complexity, micro-expression analysis is attractive but challenging, and recently becomes an active area of research. Although there are many developments in this area, a comprehensive survey that can help researchers to systematically review them is still lacking. In this survey paper, we highlight the key differences between macro- and micro-expressions, and use these differences to guide the research survey of micro-expression analysis in a cascaded structure, including neuropsychological basis, datasets, features, detection/spotting algorithms, recognition algorithms, applications and evaluation of state of the arts. In each aspect, basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, by considering the limitations in existing micro-expression datasets, we present and release a new dataset called MMEW that has more video samples and more labeled emotion types, and perform a unified comparison of representative recognition methods on MMEW. Finally, some potential research directions are explored and outlined.
Recommended citation: Xianye Ben, Yi Ren, Junping Zhang, Su-Jing Wang, Kidiyo Kpalma, Weixiao Meng, Yong-Jin Liu*. Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.