Briar Bayler — Designer
Seeing Sounds - Best Award badge 2.png

Seeing Sounds

My mother, who lost her hearing aged thirty-five, once told me she missed the sound of birds singing and the rain on the roof. This was my inspiration for my year-long conceptual research project — Seeing Sounds.

Seeing Sounds is an interactive mural walk around Wellington City that combines the magic of augmented reality (AR) technology with traditional murals to translate auditory experiences into visual animations. Seeing Sounds takes you on a journey around the city to explore different soundscapes, ‘listening’ to the environment’s sounds through an augmented reality app. With a completely new approach to sound visualisation that focuses on the ‘feel’ of sounds over auditory aspects, Seeing Sounds allows those who have lost hearing to re-experience sounds… with their eyes.

Seeing Sounds has gone on to be a finalist at the 2019 Best Awards.

If you would like to work with me to make a soundscape mural a reality — please get in touch.

 Hearing loss is one of the most common physical afflictions in the world. It is well known that being diagnosed can be scary and isolating, but the emotional impact of the loss of familiar sounds is underestimated and overlooked. Sounds many of us take for granted everyday, like waves crashing and birdsong, others would love to hear again just one more time. How might we use design to allow deaf people to ‘hear’ these sounds once again?

Hearing loss is one of the most common physical afflictions in the world. It is well known that being diagnosed can be scary and isolating, but the emotional impact of the loss of familiar sounds is underestimated and overlooked. Sounds many of us take for granted everyday, like waves crashing and birdsong, others would love to hear again just one more time. How might we use design to allow deaf people to ‘hear’ these sounds once again?

ThursPres32.png
 To find out how a wide range of people ‘see’ sounds, the designer created a co-design experiment (built on the work of Orlagh O’Brien) which looked into how a large range of people visually depict sounds through words, colour and direction. This research was then used to design visual depictions of everyday sounds.

To find out how a wide range of people ‘see’ sounds, the designer created a co-design experiment (built on the work of Orlagh O’Brien) which looked into how a large range of people visually depict sounds through words, colour and direction. This research was then used to design visual depictions of everyday sounds.

ThursPres35.png
ThursPres37.png
  Despite all linking different feelings and memories to sounds, people intuitively picked similar or the same shapes to show them. This proved that emotional sound visualisation could be done despite the subjective nature.   From these findings, a a new kind of visual communication of sounds — “emotional sound visualisation” was developed. Existing sound visualisations are rooted heavily in the scientific, such as waveforms or spectrograms. But these do not hold the ‘experience’ of hearing the sound. Through research it was found that we store sounds through our visual memories. People who had lost their hearing said that they wanted to revisit more than just the auditory aspects of sounds but they also want to ‘feel’ the sounds. Seeing Sounds set out to be one of the first sound visualisations to step into the ‘emotional experience’ space.

Despite all linking different feelings and memories to sounds, people intuitively picked similar or the same shapes to show them. This proved that emotional sound visualisation could be done despite the subjective nature.

From these findings, a a new kind of visual communication of sounds — “emotional sound visualisation” was developed. Existing sound visualisations are rooted heavily in the scientific, such as waveforms or spectrograms. But these do not hold the ‘experience’ of hearing the sound. Through research it was found that we store sounds through our visual memories. People who had lost their hearing said that they wanted to revisit more than just the auditory aspects of sounds but they also want to ‘feel’ the sounds. Seeing Sounds set out to be one of the first sound visualisations to step into the ‘emotional experience’ space.

There are five AR murals in the Seeing Sounds walk, each one visually depicting a different soundscape in Wellington.

There are five AR murals in the Seeing Sounds walk, each one visually depicting a different soundscape in Wellington.

The accompanying Seeing Sounds app has an augmented reality component that brings the sounds to life.

The accompanying Seeing Sounds app has an augmented reality component that brings the sounds to life.

As people hold their phone up to the mural, they can ‘listen’ to sounds with their eyes.

You can scan the QR code below with your phone camera to download the Eyejack app and ‘listen’ to the sounds of the city.

 In order for the user to have a full sensory experience, the AR murals are interacted with in their original environments. The city mural depicts the sounds of the bustling city, while the real city sights and smells carry on around the user as they ‘listen’. The ‘Seeing Sounds’ visualisations ‘fill the gap’ in the hard of hearing users sensory experience.  Use the Seeing Sounds app to explore Wellington and find each of the five other mural soundscapes.

In order for the user to have a full sensory experience, the AR murals are interacted with in their original environments. The city mural depicts the sounds of the bustling city, while the real city sights and smells carry on around the user as they ‘listen’. The ‘Seeing Sounds’ visualisations ‘fill the gap’ in the hard of hearing users sensory experience.

Use the Seeing Sounds app to explore Wellington and find each of the five other mural soundscapes.

 .

.

Seeing Sounds User Journey

Seeing Sounds City Mural AR Animation

Seeing Sounds is a series of interactive augmented reality (AR) murals that visualise soundscapes around Wellington city. Waves crashing at the beach, birdsong in the forest, or the chatter of conversation — translated into visuals so that those who cannot hear can once again ‘listen’… with their eyes.

The ‘City’ mural shown above depicts the conversation and footsteps that walk past as you sit in the heart of Wellington’s CBD.
You can see the AR animation here.