Puzzle-STAMPS: A Multimodal Motion-Physiology-Speech Dataset for Studying Team Collaboration and Leadership in Puzzle Solving

Studying team collaboration and leadership emergence requires multimodal datasets that combine extended, physically embodied interactions with standardized tasks and rich participant metadata. Since existing corpora typically offer only brief, sedentary interactions or narrow sets of sensing channels, we introduce Puzzle-STAMPS (Synchronized Team Analytics of Motion, Physiology, and Speech), a multimodal dataset capturing 143 participants (35 mixed-gender teams of four and one of three) engaged in a controlled "puzzle-box" experiment. Teams collaborated to solve a time-constrained cooperative game comprising thirteen physical puzzles across eight timed segments that integrate visual search, decoding, and object manipulation activities. Puzzle progression was managed via a time-based hint system to standardize team advancement while eliciting diverse leadership behaviors and coordination strategies under time pressure. We captured 76 hours of head position and orientation, torso IMU data, physiological signals (ECG, respiration, SpO2, temperature), individual audio, and room video from five viewpoints. These streams are augmented by game-state logs and standardized psychometric assessments covering personality, subjective workload, team cohesion, and leadership emergence. By combining nine sensing modalities, naturalistic noise artifacts, structured task progression, and psychometric measures, Puzzle-STAMPS provides the multimedia community with a challenging benchmark dataset for tasks ranging from leadership emergence and team performance prediction to interpersonal synchrony detection, coordination breakdown identification, and robust fusion under missing-modality conditions. Ten additional teams (39 participants) are retained as held-out evaluation data to support future benchmarks and community challenges.