MindMusic: Playful and Social Installations at the Interface Between Music and the Brain
Single- and multi-agent installations and performances that use physiological signals to establish an interface between music and mental states can be found as early as the mid-1960s. Among these works, many have used physiological signals (or inferred cognitive, sensorimotor or affective states) as media for music generation and creative expression. To a lesser extent, some have been developed to illustrate and study effects of music on the brain. Historically, installations designed for a single participant are most prevalent. Less common are installations that invite participation and interaction between multiple individuals. Implementing such multi-agent installations raises unique challenges, but also unique possibilities for social interaction. Advances in unobtrusive and/or mobile devices for physiological data acquisition and signal processing, as well as computational methods for inferring mental states from such data, have expanded the possibilities for real-world, multi-agent, brain–music interfaces. In this chapter, we examine a diverse selection of playful and social installations and performances, which explore relationships between music and the brain and have featured publically in Mainly Mozart’s annual Mozart & the Mind (MATM) festival in San Diego. Several of these installations leverage neurotechnology (typically novel wearable devices) to infer brain states of participants. However, we also consider installations that solely measure behavior as a means of inferring cognitive state or to illustrate a principle of brain function. In addition to brief overviews of implementation details, we consider ways in which such installations can be useful vehicles, not only for creative expression, but also for education, social interaction, therapeutic intervention, scientific and aesthetic research, and as playful vehicles for exploring human–human and human–machine interaction.
Detail of Publication:
Open access repository: