4D facial analysis: A survey of datasets, algorithms and applications.

Abstract: Facial information plays an important role in human communication, e.g., rich and nuanced facial expressions effectively convey emotions. Traditional data such as 2D facial images and videos are susceptible to perturbation from lighting and occlusion, while 3D static data such as mesh models lack the temporal information which is necessary to describe facial dynamics. Therefore, researches on 4D facial data (3D facial models together with time as the fourth dimension) have received considerable attention in recent years. 4D data can simultaneously reflect the complex facial temporal and space information. To fully explore these characteristics, we present a systematic overview of the 4D facial research in this paper. We give a review in terms of historical development of 4D facial datasets, acquisition process of these datasets, related algorithms and applications, and discussion of outstanding issues. We also analyze the 4D facial research works by summarizing and comparing them; in particular, we present the results on three kinds of tasks conducted on the 4D facial datasets, i.e., expression-related tasks (AU detection, expression recognition and retrieval), generation tasks (3D facial reconstruction, facial expression and facial animation generation), and other tasks (facial registration, facial recognition and facial disease diagnosis). Finally, we summarize the future research directions.

Download paper here

Recommended citation: Yong-Jin Liu, Baodong Wang, Lin Gao, Junli Zhao, Ran Yi, Minjing Yu, Zhenkuan Pan, Xianfeng Gu. 4D facial analysis: A survey of datasets, algorithms and applications. Computers & Graphics 115(2023) 423-445.