Dyadic Imitation Modeling With Lag-Aware Dual-Stream Framework for Autism Classification
Published in IEEE Transactions on Circuits and Systems for Video Technology , 2025
Abstract: This paper introduces a novel lag-aware dual-stream (LADS) framework and a carefully curated dual-view video dataset for automatic Autism Spectrum Disorder (ASD) classification through imitation tasks. In contrast to prior single-view approaches that overlook the interactive dynamics of imitation, our dataset is the first to capture synchronized experimenter-child interactions with rich pose and motion features. Building on this data, the LADS framework explicitly learns the temporal alignment between the experimenter’s demonstration and the child’s imitative response. A Lag-Aware Alignment module uses constrained cross-attention to compute an adaptive time warping and extract per-frame lag feature, revealing delays in the child’s imitation. Additionally, a lightweight diffusion-based regularizer enforces representation consistency by denoising perturbed child features conditioned on the experimenter’s motion, improving generalization. We then achieve the classification of ASD versus Typical Development (TD) behavior by integrating the aligned dual-stream features, imitation lag, and action discrepancy within an attention-pooling classifier. Experiments on our dual-view imitation dataset show that LADS significantly outperforms conventional single-stream models and a recent dyadic transformer baseline, achieving state-of-the-art classification accuracy. The results demonstrate the importance of modeling interpersonal timing in social behavior analysis. Our work provides a new, public dataset and a computational tool for interdisciplinary research, bridging computer vision and psychological studies of autism. Both dataset and code will be made publicly available.
Recommended citation:Wenqi Ji, Tong Liu, Xue Yang, Ying Guo, Xin Chen, Pei Yang, Kaiyun Li, Yong-Jin Liu.Dyadic Imitation Modeling With Lag-Aware Dual-Stream Framework for Autism Classification. IEEE Transactions on Circuits and Systems for Video Technology ( Volume: 36, Issue: 5, May 2026), https://doi.org/10.1109/TCSVT.2025.3642987.