Hearing loss is one of the most common physical afflictions in the world. It is well known that being diagnosed can be scary and isolating, but the emotional impact of the loss of familiar sounds is underestimated and overlooked. Sounds many of us take for granted everyday, like waves crashing and birdsong, others would love to hear again just one more time. How might we use design to allow deaf people to ‘hear’ these sounds once again?
To find out how a wide range of people ‘see’ sounds, the designer created a co-design experiment (built on the work of Orlagh O’Brien) which looked into how a large range of people visually depict sounds through words, colour and direction. This research was then used to design visual depictions of everyday sounds.
Despite all linking different feelings and memories to sounds, people intuitively picked similar or the same shapes to show them. This proved that emotional sound visualisation could be done despite the subjective nature.
From these findings, a a new kind of visual communication of sounds — “emotional sound visualisation” was developed. Existing sound visualisations are rooted heavily in the scientific, such as waveforms or spectrograms. But these do not hold the ‘experience’ of hearing the sound. Through research it was found that we store sounds through our visual memories. People who had lost their hearing said that they wanted to revisit more than just the auditory aspects of sounds but they also want to ‘feel’ the sounds. Seeing Sounds set out to be one of the first sound visualisations to step into the ‘emotional experience’ space.
There are five AR murals in the Seeing Sounds walk, each one visually depicting a different soundscape in Wellington.
The accompanying Seeing Sounds app has an augmented reality component that brings the sounds to life.
As people hold their phone up to the mural, they can ‘listen’ to sounds with their eyes.
You can scan the QR code below with your phone camera to download the Eyejack app and ‘listen’ to the sounds of the city.
In order for the user to have a full sensory experience, the AR murals are interacted with in their original environments. The city mural depicts the sounds of the bustling city, while the real city sights and smells carry on around the user as they ‘listen’. The ‘Seeing Sounds’ visualisations ‘fill the gap’ in the hard of hearing users sensory experience.
Use the Seeing Sounds app to explore Wellington and find each of the five other mural soundscapes.
Seeing Sounds User Journey
Seeing Sounds City Mural AR Animation
Seeing Sounds is a series of interactive augmented reality (AR) murals that visualise soundscapes around Wellington city. Waves crashing at the beach, birdsong in the forest, or the chatter of conversation — translated into visuals so that those who cannot hear can once again ‘listen’… with their eyes.
The ‘City’ mural shown above depicts the conversation and footsteps that walk past as you sit in the heart of Wellington’s CBD.
You can see the AR animation here.