Abstract
3D human motion generation driven by text and speech is vital for VR and animation, but it is constrained by scarce 3D motion capture (MoCap) data and inconsistent evaluation metrics. To address these issues, this paper proposes a framework that integrates two key advances: text-speech multimodal guidance (inspired by T3M, Peng et al., 2024) and 2D motion priors (adapted from Motion-2-to-3, Pi et al., 2024), with standardized evaluation via the CLaM library (Chen et al., 2024). Specifically, we fuse BERT-extracted text features and MFCC-based speech features to capture semantic-temporal cues, then inject 2D motion priors (learned from large-scale 2D datasets) to mitigate 3D data scarcity. Evaluation via CLaM’s metrics (PCK@0.1, CLIP score, jerkiness) on Human3.6M and CMU Mocap shows our method outperforms T3M by 12.3% in motion similarity (PCK@0.1) and 13.0% in text-motion alignment (CLIP score). This work demonstrates that 2D priors synergize with text-speech guidance to boost 3D motion quality, while CLaM ensures objective performance comparison.
