Experiencing sound through interactive jewellery and fashion accessories

In this paper we report the results of a series of experiments conducted as part of a participatory Research through Design (RtD) project [1] which aims to develop interactive jewels for deaf women. The project moved from the idea of bringing mindful attention to aesthetics in designing for disability, exploring cross-modal sensory associations. Our jewels detect ambient sound and represent them with light patterns, vibration patterns, and shape change. Preferences can be set through an app that allows to record sounds of interest (e.g. doorbell, a household appliance) and to notify them through visual or tactile feedback of the jewel. In order to design meaningful cross-modal sensory associations between sounds in input and behaviours of the jewels in output, we engaged hearing and deaf people in an experiment on the association of light patterns to sound types. Preliminary results show the difficulty in associating light patterns to sound types without listening to the sound during the task. On the contrary, when the sound was preliminary heard, there was a higher possibility to associate it with the correct light pattern. Even if the results of the experiment are preliminary due to the small sample size of participants and the limited number of sounds used in the study, nevertheless they gave us insights about the design of the jewelry system and for follow up studies.