Authors:

Patrick O’Toole, Donald Glowinski, Maurizio Mancini

Publication Type:

Refereed Conference Meeting Proceeding

Abstract:

Can cross-modal associations between notes and colours and between notes and emotions be strengthened by using an ad-hoc designed computer program? In this paper we present an experiment that we conducted to see (H1) if participants’ associations between the auditory and visual senses could be strengthened through immersion in a multi-sensory program and (H2) if there was any correlation between auditory-visual and the auditory-emotion associations. In our experiment, participants were asked to use the SoundStrokes program, an application that translates notes played on an analog music instrument (i.e., a guitar) into coloured graphical shapes (i.e., rectangles). Between several training sessions with the program, participants were asked to associate single music notes to colours and emotions. Applications of our work may vary from entertainment, to enrich in a systematic manner the user emotional experience in, for example, videogames or immersive experiences, to digital health, in which relatively simple tools can be developed, for example, to reduce anxiety through music.

Conference Name:

8th International Conference on Affective Computing and Intelligent Interaction (ACII 2019)

Digital Object Identifer (DOI):

10.1109/ACII.2019.8925465

Publication Date:

03/09/2019

Conference Location:

United Kingdom (excluding Northern Ireland)

Research Group:

Personal Sensing

Institution:

National University of Ireland, Cork (UCC)

Open access repository:

No