Technical Note: Quantifying music-dance synchrony with the application of a deep learning-based 2D pose estimator

Abstract

Dance interventions are more effective at improving gait and balance outcomes than other rehabilitation interventions. Repeated training may culminate in superior motor performance compared to other interventions without synchronization. This technical note will describe a novel method using a deep learning-based 2D pose estimator: OpenPose, alongside beat analysis of music to quantify movement-music synchrony during salsa dancing. This method has four components: i) camera setup and recording, ii) tempo/downbeat analysis and waveform cleanup, iii) OpenPose estimation and data extraction, and iv) synchronization analysis. Two trials were recorded: one in which the dancer danced synchronously to the music and one where they did not. The salsa dancer performed a solo basic salsa step continuously for 90 seconds to a salsa track while their movements and the music were recorded with a webcam. This data was then extracted from OpenPose and analyzed. The mean synchronization value for both feet was significantly lower in the synchronous condition than the asynchronous condition, indicating that this is an effective means to track and quantify a dancers movement and synchrony while performing a basic salsa step.