We interview speech therapist Laura Roig about Audivers 360º, the winning project of the GrausTIC Award for Social Integration through Digitalization awarded by the iSocial Foundation in 2023

How did the Audivers 360º project come about?

The Audivers 360º project was born at the beginning of 2023 as a result of the Harvi project. This project consisted of speech therapy sessions aimed at deaf children who used virtual reality, using 3D glasses. At that time, the advantages of doing speech therapy with virtual reality to work on sounds from everyday life were beginning to be discovered.

The main disadvantage of the Harvi project was the weight of the glasses, especially in children. On the other hand, this therapy generated a disconnection with the environment, since wearing the glasses meant a loss of relationship with others and with the professional who accompanies the person in the room.

Taking the Harvi project as a reference and thanks to Suara Cooperativa, which was just starting to do projects with immersive reality, we met Broomx, the company that owns the tool. This allowed us to begin developing immersive reality as a tool for auditory rehabilitation of children and people with deafness.

How was the process of giving shape to the project like?

In July 2023, we began a pilot test with adults with deafness, where we tested immersive scenarios that had already been created by Broomx. However, when conducting the pilot test, we identified several drawbacks and the need to adapt the scenarios to people with deafness.

The scenarios we had were very empty in sound content and, since the objective was to work on auditory skills, we needed the sound environment to be richer. Then, we co-created three new scenarios: the Boqueria Market and Plaça Espanya, which are adaptations of existing scenarios and a children’s playground, which was new. In these scenarios, we added ambient sound, as a masking of the sounds we wanted to work on

In February 2024, we conducted another pilot test with the new scenarios, and this time they did respond to the needs of the users. We secured funding to build an immersive room here at ACAPPS during April and May 2024. Since then, we have fully implemented the tool for all people undergoing hearing rehabilitation at our facilities.

What needs does the project respond to?

The Audivers360 project allows us to place the person at the center of the therapy and work specifically in those real environments that generate the most difficulties for them.

It mainly responds to the need that adults and children with deafness have to work on real situations in a safe environment. Immersive reality allows us, for example, to reproduce the sound of a car and ask the person what they are listening to. In addition, we can decide whether or not we want to show the car at that moment. If the person sees the car while hearing the sound, this reinforces auditory discrimination with visual support. On the other hand, if they only hear the sound without seeing the car, they have to remember that that noise corresponds to a car, which favors the work of auditory memory.

What user profile do you work with?

We mainly use the tool to work on auditory skills with deaf people who wear hearing aids, such as an implant or hearing aids. Right now, most people who do auditory rehabilitation with us are adults with deafness, but it is true that we are increasingly starting to incorporate children, although for reasons of extracurricular activities or family reconciliation it is more complicated for them to travel here.

We also work with older people with hearing loss. In this case, the tool helps to promote more active aging. Older people often do not have much access to new technologies, and these sessions motivate them and they perceive a faster evolution of their progress: they tell us that it is easier for them to talk to the person selling the newspaper or that they can hear a car approaching them.

What are the advantages of immersive speech therapy?

Immersive speech therapy allows us to work with real-life sounds and situations in an integrative, enjoyable and safe way, avoiding auditory overattention. Working with real sounds and situations in a controlled way brings naturalness and realism to the therapy. Unlike sessions without technological innovation, immersive therapy helps people with deafness to better understand what is happening around them, which is a stimulus for auditory comprehension because it is not only about identifying a sound, but also recognizing it. Identifying a sound is only the first step; then you need to know what it represents and, finally, understand its meaning. It is not the same to hear the sound of a door, know that it is a door, and associate this sound with the word “door”. The use of immersive reality reduces the error rate in activities. On the other hand, immersive therapy also helps to work more on auditory memory, because we can work on auditory input and visual input at the same time.

How is customization worked on in this project?

We do individual sessions of 45 minutes per week. Especially in terms of auditory rehabilitation, it is interesting that we can do them individually. This way we can more specifically address the needs of each person undergoing therapy.

Before a person starts at Audivers360, we do a speech therapy assessment session, to find out the person’s needs and above all to know how they are in terms of hearing and, with that, plan the work plan. From there, and depending on their interests, we work on various aspects of hearing with the immersive scenarios. For example, since it is stereo, with two speakers in front (right and left) and one in the back, the immersive room allows us to lateralize sounds, a fundamental aspect in rehabilitation.

How does this tool help improve the communication experience of people with deafness?

With Audivers360, we work on both auditory skills, which are the identification, recognition and understanding of sounds and words, and verbal expression. For example, the speech therapist tells a story related to the immersive scenario that is being seen and the user listens to this story, and this allows us to work on auditory comprehension. Later, we ask them to tell the same story again, and in this way we work on verbal expression.

In which environments can this tool be used?

We have the room at the ACAPPS premises. Currently, the attention is very centralized in Barcelona, and usually the adults, children and families who participate in the sessions travel from different points of the Metropolitan Area.

However, the projector we use is small and can be transported. For example, if we wanted to do a session in the hospital, we would only have to find a room with white walls.

Speech therapy session in the ACAPPS immersive room

How has Audivers been received by users?

During the pilot test, we asked people to tell us, using a visual analog scale, how they felt, from 1 to 10, before the session and how they were when they left. With this, we saw that emotional well-being with the immersive room really increased, there was a positive evolution. Currently, in most cases, we carry out satisfaction surveys at the end of each activity and what they tell us is that Audivers360 provides them with a new, more motivating and enjoyable experience. On the other hand, immersive speech therapy also allows us to work on executive functions, such as memory and attention. Most people tell us, through satisfaction surveys, that with Audivers360 the experience is more motivating and that it does not produce as much auditory fatigue as traditional sessions, which greatly increases the desire to continue with the rehabilitation.

What does it mean to you to have won the Catalonia Graustic Digital Innovation Awards in 2023?

Winning the Innovation Award made us very excited, not so much because of the fact of winning, but because of the recognition of knowing that we are doing work that has a social impact and that we are contributing to improving the lives of people with deafness. It is also important because we are developing an innovative proposal in terms of technology, at a time when technology is advancing at breakneck speed, for example with artificial intelligence. In this context, receiving this recognition as a pioneering and innovative project makes us very excited and makes us aware of the value of the work that the entire technical team behind it does.

What are your current future challenges?

Beyond the technical improvements of Audivers, our challenge is to publicize the project. We would like it to be a methodology that could also be incorporated into hospitals or other centers and to publicize it throughout the healthcare sector, so that it could be used by otolaryngologists, speech therapists, psychologists, etc.