Sensorimotor Synchronization in Embodied Rhythmic Agents
Abstract
This thesis aims at modeling rhythmic collaboration to develop algorithms that can be implemented as an “embodied SMS agent,” an agent capable of coordinating rhythmically with its environment in a human way.
This work fits within the tradition of sensorimotor synchronization (SMS), i.e., the study of coordination between perception and action, and employs theoretical frameworks, experimental data, and simulation. It builds on previous studies in rhythmic collaboration over delayed networks and continues with experiments using human-computer interaction to study human response to rhythmic stimuli. It also uses behavioral studies to simulate human functions at rhythmic tasks.
We will present a rigorous formulation of an embodied SMS agent and its sensorimotor coupling with the environment, accounting for the so-called mind/body/environment triad. Then, we show that the same framework that a physical robot uses to navigate its space spatially can describe an agent capable of rhythmic coordination. Two different attempts to design the “mind” of an SMS agent will be introduced. One is a black-box approach by inferring rhythmic behavior merely from the step response, and the latter is a bio-inspired behavioral SMS according to the “first principles.” The limits of each model are investigated.
In this work, the “Chafe Effect,” i.e., the acceleration of mutual rhythmic handclapping collaboration at minimal latencies, is linked to the notion of negative mean asynchrony (NMA), the tendency to precede tones in tapping to a metronome. The dependencies of NMA on the balance between auditory and tactile is also studied, showing that tactile (as opposed to auditory) afferent dominance produces a smaller negative mean asynchrony.
We have provided a formulation to quantify the strategies taken by performers in rhythmic collaboration and argued that this approach could offer an alternative or addition to the more deterministic dual error correction mechanisms in SMS, namely period and phase error correction. A particular example of such strategy was discussed as a “delay compensation factor” to capture the voluntary choice of preempting taps to received tones in the context of the Chafe effect.
A novel approach borrowed from cybernetics and control theory is introduced to SMS. The pole-zero model helps identify rhythmic coordination properties by the parameters of a rational transfer function described in the complex frequency domain. This model is shown to provide behavioral clues to different processes behind responding to subliminal and supraliminal changes in the tempo, possibly due to different neural correlates to such brain mechanisms. A two-pole, single zero with a delay, or P2DUZ, was found helpful in modeling humans’ response to stepchanging metronomes.
Also, using neurophysiological references of rhythm and based on behavioral studies, an SMS agent capable of rhythmic coordination is simulated. The behavioral/neurophysiological SMS agent is shown to produce results comparable to those observed in human subjects by the authors and others.
At last, a design concept for a potential application of the model, a “virtual musical co-performer” for automated musical accompaniment, is presented.