Interactive Music Generation System Using Unconstrained Body Motion

We are developing a sound interaction system that generates musical sounds from unconstrained body motions. Although many musicians and researchers have investigated musical expressions through body movements, to generate musical sounds, composing the music and playing the piece correctly is still required. In this study, by overlapping the looped-playing of simple motifs, it is possible to generate musical sounds in real time and interactively by simply moving the body freely in front of the camera. This system allows a wide range of users, from beginners to experts, to experience a new way of generating music through body motion with no prior training.