51³Ô¹ÏÍø

51³Ô¹ÏÍø researchers have won a €5.7million EU Horizon 2020 grant to develop AI-informed immersive audio techniques.

51³Ô¹ÏÍø researchers have won a €5.7million EU Horizon 2020 grant to develop AI-informed immersive audio techniques.

The project, called , will see researchers develop immersive 3D sound for virtual and alternative reality situations like online meetings, lectures, and gaming.

Our technology could help users have more real, more immersive experiences that convey the nuanced feelings and intentions of face-to-face conversations.

Dr Lorenzo Picinali. Dyson School of Design Engineering

Immersive audio creates sounds that are perceived as coming from different directions, to emulate how we experience audio in the real world. Although similar technologies are used in spaces like cinemas, virtual spaces like video chat, gaming, and doctor appointments lack this kind of technology.

During the  where many are working from home, emulating real-life scenarios more accurately could help rebuild the conversational nuances and social cues that can be lost during online communication.

SONICOM will use artificial intelligence (AI) to make online spaces more realistic – by, for example, having colleagues sound like they are speaking to your right or left, possibly also at different distances, like in a real meeting space.

Lead investigator Dr Lorenzo Picinali, of 51³Ô¹ÏÍø’s Dyson School of Design Engineering, said: “Our current online interactions are very different to real life scenarios: social cues and tone of voice can be lost, and all the sound comes from one direction. Our technology could help users have more real, more immersive experiences that convey the nuanced feelings and intentions of face-to-face conversations.”

This is one of the first times AI will be used to personalise sound. Everyone perceives 3D, or spatial, sound differently thanks to the particular shape of their ears and head, and so high-quality simulations must be uniquely tailored to each individual.

The researchers will employ AI and a data-driven approach to link the physiological characteristics of the ear to the individual acoustic filters which allow people to hear where sound is coming from. This will provide personalised audio spatialisation from only a small amount of data acquired from users.

Imagine a virtual meeting space where you see colleagues to your right, left, and across from you. We want to make this possible in audio form.

Dr Lorenzo Picinali. Dyson School of Design Engineering

They will also alter spatial sound rendering depending on its context. Doctor and therapy appointments might sound ‘closer up’ for greater intimacy, whereas lecturers and talks could sound farther away to emulate the real-world lecture experience.

Once the personalised immersive audio techniques have been developed, the researchers will explore, map and model how their use influences listeners’ behaviour and physiology during social interactions.

Dr Picinali said: “Imagine a virtual meeting space where you see colleagues to your right, left, and across from you. We want to make this possible in audio form, using AI not only to improve and personalise sound, but also to the reactions of the listeners and predict how this could influence the conversation.”

They hope to have developed the techniques within three years and will make them and the accompanying AI available to researchers and companies who want to use them in new hardware.

The tech could help virtual meetings feel realer.

The project includes researchers from seven research and academic institutions and three companies across six European countries: 51³Ô¹ÏÍø (UK),  (France),  (Austria), (Italy),  (Greece),  (Spain),  (UK), (Austria), (UK), and  (Austria).

Dr Picinali and the SONICOM consortium will be part of a cross-collaborative initiative with three other projects awarded under the same EU Horizon 2020 call known as ‘Artificial Intelligence for extended social interaction.’ The aim is to identify synergies within their projects and refine a joint vision to maximise the impact of each project’s research in their respective emerging technological paradigms.

SONICOM builds on several previous 51³Ô¹ÏÍø research projects such as the , also funded by EU Horizon 2020 and coordinated by the Audio Experience Design team at 51³Ô¹ÏÍø. 3D Tune-In set the grounds for the development of advanced techniques and tools for immersive audio in virtual reality, and SONICOM is taking this initiative forward at a larger scale, engaging with most of the major players in the field within Europe.

By Caroline Brogan

The immersive audio technology is developed by SONICOM, a new EIC Pathfinder project, which will revolutionize the way we interact socially within AR/VR environments and applications.

First published with the  (opens in a new window)

The immersive audio technology is developed by SONICOM, a new EIC Pathfinder project, which will revolutionize the way we interact socially within AR/VR environments and applications.

 is a 60-month project, which started in January 2021. It was selected among proposals submitted for FET Proactive Emerging Paradigms and Communities call (), in subtopic A: Artificial Intelligence for extended social interaction.

With the help of Artificial Intelligence (AI) and data-driven technological paradigms, the SONICOM project will transform auditory social interaction and communication in Virtual and Augmented Reality (VR&AR). It focuses on immersive audio technologies, which could eliminate the different hearing experiences we feel between physical and remote communications. The SONICOM team presents the idea behind the project, as follows:

“Picture yourself being able to dynamically change the position of the various participants within a virtual conversation, modifying also the acoustical characteristics of the simulated environment. Then extend this to an interaction where some participants are present in person in the same environment, and some are accessing it remotely; imagine ‘blending’ the real and virtual so that it is not possible, from an auditory point of view, to distinguish between the two.”

Lead investigator Dr Lorenzo Picinali, of 51³Ô¹ÏÍø’s Dyson School of Design Engineering, said: “Our current online interactions are very different to real life scenarios: social cues and tone of voice can be lost, and all the sound comes from one direction. Our technology could help users have more real, more immersive experiences that convey the nuanced feelings and intentions of face-to-face conversations.”

In the first phase, the SONICOM team of researchers and creative tech experts from across Europe  will design a new generation of immersive audio technologies and techniques to transform social interactions, specifically looking at customisation and personalisation of the audio rendering. The researchers will explore and analyse behavioural, physiological, kinematic, and psychophysical reactions of listeners within social interaction scenarios, in order to develop appropriate hardware and software proofs of concept.

SONICOM is part of the paradigm of emerging virtual technologies. Over the five-year project, the team aim to release a comprehensive ecosystem for auditory data closely linked with model implementations and immersive audio rendering components, reinforcing the idea of reproducible research, and promoting future development and innovation in the area of auditory-based social interaction. Dr Picinali said:

“Imagine a virtual meeting space where you see colleagues to your right, left, and across from you. We want to make this possible in audio form, using AI not only to improve and personalise sound, but also to the reactions of the listeners and predict how this could influence the conversation.”

Alongside this work, the SONICOM consortium will be part of a cross-collaborative initiative with three other projects awarded under the same Horizon 2020 call known as ‘Artificial Intelligence for extended social interaction.’ The aim is to identify synergies within their projects and refine a joint vision to maximise the impact of each projects’ research in their respective emerging technological paradigms.

The SONICOM project brings together 10 experienced teams from 6 European countries (United Kingdom, France, Italy, Austria, Greece and Spain). The coordinating institution is .

Background information

 and  are now part of the  (specifically the Pathfinder), the new home for deep-tech research and innovation in , the EU funding programme for research and innovation.

With representatives from all partners present, the meeting provided a fruitful basis for discussions on delivering the project’s pioneering objectives over the next 5-years.

SONICOM officially kicks off with the inaugural annual meeting

With representatives from all partners present, the meeting provided a fruitful basis for discussions on delivering the project’s pioneering objectives over the next 5-years.

The SONICOM Kick-Off Meeting took place online on Thursday, 20th May 2021. With a lot to get through, the condensed agenda began with Coordinating partner and Principal Investigator, Dr Lorenzo Picinali from 51³Ô¹ÏÍø, chairing salutations and introductions. The consortium was fortunate to be joined by the Advisory Board and External Ethics Advisor who were introduced to partners and their teams involved in SONICOM.

Despite the tight schedule, Dr Picinali had time to kick off the presentations with a quick pop quiz; posing a series of questions to the consortium on head-related transfer functions (HRTFs) which sparked interesting discussions exploring what answers SONICOM research is hoping to uncover in these areas.

Following this, the Work Package (WP) presentations were the focus of the remaining morning session – with each WP leader sharing a description of work they plan to achieve and research questions they hope to investigate. The penultimate session was dedicated to conversations with the Advisory Board which encapsulated the proverbial phrase, so many things to discuss, so little time. Nevertheless, the Board and the consortium covered a lot of ground in areas such as impact and identifying commercial collaborative partners.

The last part of the meet held a series of bilateral meetings with the members of the ,  and  consortia, forming part of our commitment to the Cross-Projects Collaboration initiative set for all ‘’ projects. The projects honed in on prospective areas for collaboration and reciprocal activities to maximise each other’s research outputs including public outreach activities and technology integration.

The meeting wrapped up Thursday early evening with a close and ‘next steps’ from Dr Picinali. Overall, the meeting was a huge success, and we look forward to collaborating with such an excellent consortium and the FETPROACT-EIC-07-2020 projects over the project’s 5-year lifetime.

SONICOM'S coordinator Dr Lorenzo Picinali (51³Ô¹ÏÍø) in collaboration with the University of Malaga have developed a set of binaural software tools. The fruit of their labour was the 3D Tune-In Toolkit, and they have made it freely available for all non-programmer binaural enthusiasts

Original article published on 

SONICOM’s coordinator, Dr Lorenzo Picinali () in collaboration with partners,  have developed a set of binaural software tools, for free. Originally developed as a research tool from , a European Union’s Horizon 2020 research project, the people behind it have considered it such a useful bit of programming that they’re essentially giving it away for free!

The algorithm itself has been released as an open-source resource, and is available on GitHub, but non-programmer binaural enthusiasts can get in on the action too, thanks to a number of self-contained software tools that employ the same algorithm. For example, the Test Application programme is essentially a stand-alone spatialiser, into which you can load up mono WAV files and move them around a virtual space. Usefully, the Test Application speaks OSC (Open Sound Control), so using a phone, computer or tablet running one of the many freeware OSC apps around, you can perform real-time head tracking, and whizz your mono audio sources about in the virtual 3D space (see video below).

SONICOM researchers dive-in to understand more about the values and safety of using AI in the project.

SONICOM researchers dive-in to understand more about the values and safety of using AI in the project.

“Your scientists were so preoccupied with whether or not they could that they didn’t stop to think if they should.” The famous line from  (Jeff Goldblum) in Jurassic Park is a regular reference when discussing new scientific and technological frontiers, especially none more so in the context of artificial intelligence. As part of a recent workshop they participated in, SONICOM researchers did not exclusively tackle what Dr Malcom says, rather they assessed the ethical dilemmas around the tools and applications that will be developed in SONICOM over the 5-year project.

The “Ethics and AI in SONICOM” workshop was led by SONICOM’s external ethics advisor, Dr Catherine Flick; an ethicist and Reader in Computing and Social Responsibility at De Montfort University. It centred on completing  tool which was developed only recently (launched in 2020) by the AI , or AI HLEG for short, who were appointed by the European Commission to provide advice on artificial intelligence strategy. The ALTAI tool is aimed to support projects such as SONICOM in assessing the trustworthiness of AI in their systems, and SONICOM’s researchers dived right in.

SONICOM researchers during the virtual workshop exploring the ALTAI tool

The consortium identified and evaluated the values and principles behind the way AI will be leveraged to design novel immersive audio technologies and techniques. In evaluating what risks the systems using AI might generate and the potential impacts on clusters such as society, the environment, the consumer etc, the workshop enabled SONICOM’s researchers to think about ‘responsible competitiveness’. SONICOM’s researchers considered through completing this tool the ways in which the design, development, and use of AI in the project can be trusted by end-users.

In speaking with Dr Flick, she notes that “the SONICOM team had a productive discussion around the AI systems being implemented within their project using the ALTAI self-assessment tool for trustworthy AI.” Also, engaging the consortium in thinking about minimising risk as the project progresses, Dr Flick said that  “while the project is very sound in its risk mitigation strategies, a number of future tasks were identified to ensure the trustworthiness of their system with its end-users and the general public.” Overall, having trustworthy AI that is lawful, ethical, and robust.

The consortium will continue to assess AI and ethics as the project progresses.

More information on the ALTAI tool is available on the .

More about Dr Catherine Flick’s work can be found on .

Learn about the Aural Assessment By means of Binaural Algorithms (AABBA) group, whose goal is to promote exploration and development of different models of human hearing and their applications.

Developing models of human hearing is an area of research with many applications, from improving hearing aids and implants to creating immersive audio for virtual reality, and connecting researchers working in this space from around the globe is vital to making progress.

Introducing AABBA – not the latest Swedish pop group, but a group of scientists that collaborate on the development and applications of different models of human hearing.

Set up in 2009 by Professor Jens Blauert as a way for himself and other acoustics research colleagues to discuss joint projects and work, the  has since grown to nearly 30 regular members from across 14 countries.

SONICOM member Dr Piotr Majdak has since taken the reins from Professor Blauert on AABBA’s Executive Board.

“I’m very proud to be taking on the leadership [of AABBA],” said Majdak, “But I couldn’t do it without the help of my colleagues and friends.”

A space for making connections

A core pillar of AABBA is building networks and connections for people working in acoustics research.

AABBA members meet annually in Vienna for open discussions and presentations on their research. They especially encourage members to bring in students and young scientists associated with their projects to the meetings to present the research they’re working on, meet people, and exchange ideas.

Together, the network has resulted in in joint publications and special sessions at international conferences. Recently they have worked together to create a special issue in , where, despite an open call, almost all chapters have been contributed by AABBA members.

It has also provided validated (source) codes for published models of binaural and spatial hearing to our collection of auditory models, known as the auditory modelling toolbox (AMT). The AMT has since received an update to the , and contains 60 models and over 40 GB of auxiliary data and cached modelling results are available for download on the fly.

If you are interested in getting involved with AABBA or joining the network, get in touch with Dr Piotr Majdak.

Upcoming session and International Congress on Acoustics 2022

AABBA will be putting on a Special Session titled “Spatial Hearing” at the International Congress on Acoustics (ICA) 2022 in Gyeongju, South Korea.

Find out more about the conference .

Members of the SONICOM team attended and presented at the UK Hearing, Audiology & Sciences Meeting 2022.

Members of the SONICOM consortium had the pleasure of attending and presenting at the UK Acoustics Network (UKAN) sponsored Hearing, Audiology & Sciences Meeting this week on the 12 and 13 September in Southampton, UK.

The conference was open to anyone with an interest in the science of hearing, audio and audiology, with the aim of facilitating a friendly exchange of work and ideas.

Dr Lorenzo Picinali, scientific coordinator of SONICOM, chaired a session on spatial hearing and immersive audio rendering, which saw several members of 51³Ô¹ÏÍø’s  present their work in this space.

“It was an excellent venue for presenting some early results from our work, as well as some plans for future works, and get very valuable feedback in a very informal way!” said Dr Picinali.

This session included a talk from Rapolas Daugintis, a PhD student in the team, who discussed his work on the development and evaluation of auditory-model-aided non-individual HRTF selection procedure.

“This was my first time presenting my work as a PhD student in an academic conference format. It was a good platform to develop my science communication skills in quite a relaxed environment and meet the academic community,” said Daugintis.

The meeting also had a poster session, which included Dr Thibault Vicente displaying work on assessing the effect of head-related transfer function on spatial stream segregation.

“This poster was a good opportunity to present our pilot study. It is always relevant to have feedback from other researchers at this stage, so that the protocol can be adjusted before the final data collection,” said Dr Vicente.

See the full programme from the meeting .

The book brings together experts in spatial audio for virtual reality to tackle the design of 3D spatial interactions in an audio-centred and audio-first perspective.

A new SONICOM-sponsored book titled “” has now been published.

Edited by SONICOM PI Professor , University of Padova and 51³Ô¹ÏÍø, and Professor Stefania Serafin, Aalborg University, the book brings together experts in spatial audio for virtual reality to tackle the design of 3D spatial interactions in an audio-centred and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences.

“The basic and most obvious assumption that motivates this volume is: it is hard to live in a world without sound and it is hard in virtual environments (VEs) too,” says Professor Geronazzo. “The good news brought by this book is that Virtual Reality finally sounds plausible. Advances in several fields are now able to provide an immersive listening experience that is perceptually indistinguishable from reality which means that immersive sounds could make interaction intrinsically natural.”

Since 2015, the editors have organised the IEEE Virtual Reality workshop series “” with the mission of increasing the awareness of the importance of sonic elements when designing immersive virtual environments among the virtual reality community and junior researchers.

This volume aims to provide an organised starting point on which to develop a new generation of immersive experiences and applications. Contributing authors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. This was designed to give to the reader a broad view and a clear introduction to state-of-the-art technologies and design principles, and to the challenges that might be awaiting us in the future.

Through an overview of emerging topics, theories, methods, tools, and practices in sonic interactions in virtual environments research, the book aims to establish the basis for further development of this new research area.

User and system adaptation in binaural audio

SONICOM PIs Dr  and Dr  together wrote a chapter of the book titled “System-to-User and User-to-System Adaptations in Binaural Audio”.

The chapter concerns concepts of adaption in a binaural audio context (i.e. headphone-based 3D audio rendering and associated spatial hearing aspects), considering first the adaptation of the rendering system to the acoustic and perceptual properties of the user, and second the adaptation of the user to the rendering quality of the system.

Their discussions cover an overview of the basic mechanisms of human sound source localisation as well as more complex concepts and processes, such as HRTF selection (system to-user adaptation) and HRTF accommodation (user-to-system adaptation). The potential of these two approaches are discussed, considering their combined use in a practical context, as well as introducing a few open challenges for future research.

SONICOM’s sponsorship to the project also contributed to its full transition to open access, and the full book can be accessed for free .

SONICOM took part in the Royal Society's Summer Science Exhibition, engaging hundreds of people with immersive audio across the week.

From 4-9 July, SONICOM exhibited at the Royal Society’s Summer Science Exhibition 2023, one of the largest science engagement festivals in the UK.

As the Royal Society’s flagship event, the Summer Science Exhibition attracts thousands of visitors every year, from families with 4-year-olds to school groups to eminent Fellows of the Royal Society.

The ‘Virtual Audio: Illusion or Reality?’ exhibit led by Lorenzo Picinali, a member of the Audio Engineering Society, featured research projects from Picinali’s Audio Experience Design team at 51³Ô¹ÏÍø, including SONICOM and the NIHR-funded BEARS project.

The exhibit focused on how people and technology can adapt to each other to enhance virtual and physical interactions.

Visitors to the exhibit got the opportunity to learn about SONICOM through chats with members of the team and hands-on activities, including having their own ears 3D scanned to learn about how morphology needs to be taken into account when personalising audio.

“It was a lot of fun chatting to the public about our research and meeting incredibly interested people from all walks of life,” says Dr Katarina Poole, a SONICOM researcher based at 51³Ô¹ÏÍø. “There were a lot of discussions on how immersive audio could be used – why it’s not used, why it should be – which was good to expand my thinking on how my and the lab’s research could be used.”

Find out more about the research featured at the Exhibition on our .

You can also watch some of the highlights of the week in the video below:

Last month, members of the SONICOM team presented at the 10th Convention of the European Acoustics Association – Forum Acusticum 2023.

Last month, members of the SONICOM team from 51³Ô¹ÏÍø, the Austrian Academy of Sciences, Sorbonne University and the University of Malaga attended the 10th Convention of the European Acoustics Association –  – in Torino, Italy.

Team members presented a variety of work being undertaken as part of SONICOM, including the new Spatially Oriented Format for Acoustics (SOFA) release, as well as research on HRTF individualisation, accommodation and upsampling.

“It was great to be able to showcase the variety of work going on within SONICOM,” said Dr Lorenzo Picinali, SONICOM lead investigator. “The SONICOM team continues to go from strength to strength, and the work we presented at Forum Acusticum is just a taste of what’s to come.”

SONICOM researcher, Dr Aidan Hogg, presenting at Forum Acusticum 2023

Explore some of the research presented:

Explore the  on the Forum Acusticum website.

Members of the SONICOM consortium gathered with external researchers and stakeholders to exchange knowledge and ideas.

Representatives from across the SONICOM consortium gathered at the University of Glasgow on 20 and 21 November to take part in the SONICOM Research Sandpit – a chance to exchange knowledge and experiences between the project consortium and external researchers and stakeholders.

The SONICOM team were joined by four guest speakers who gave their insights into current challenges in auditory research, including head-related transfer function (HRTF) personalisation as well as augmented reality and acoustics simulation.

The guests included:

• Dr Fabian Brinkmann (Technical University of Berlin), who discussed how we can improve the realism of simulated HRTFs, including taking into account the impact of varying head-torso angle, as well as hair and skin.
• Dr Archontis Politis (Tempere University), who presented on his research in parametric spatial audio processing and spatial acoustic scene analysis.
• Dr Annika Neidhardt (University of Surrey), who presented her work on the perceptual optimisation of room acoustics for AR/MR.
• Dr Enzo De Sena (University of Surrey), who discussed real-time room acoustic rendering for AR/MR/VR.

The team took full advantage of being together in person along with our external guests to have some productive and insightful discussions on topics ranging from newly designed methods for assessing HRTF personalisation, to explorations on the impact of room acoustic processing choices on complex interactions.

After an initial overview of the achievements and future plans of the SONICOM project, the invited guests presented their research engaging in relaxed and informal discussions with the consortium members, looking at joint ideas for future research, possible collaborations, as well as common efforts for tackling the various open challenges.

“It was encouraging to see how researchers from multiple disciplines and from inside and outside the SONICOM project teamed up not only to discuss and tackle challenges of future immersive audio systems but also to make their work available to the scientific community and general public,” said Dr Brinkmann.

Plans for SONICOM’s  were also further developed – keep an eye on our  and website for updates on when the Challenge will be accepting submissions.

The SECTG cluster joined together at 51³Ô¹ÏÍø for a transformative event designed for aspiring researchers.

On 23rd November, the SECTG cluster joined together at 51³Ô¹ÏÍø for a transformative event designed for aspiring researchers. The one-day workshop was focused on bringing the cluster together to foster collaboration, innovate, provide a platform to present work, exchange ideas, and network with fellow scholars from diverse fields.

The SECTG cluster is an alliance of five European sister projects funded under the Horizon 2020 FET PROACT-EIC-07-2020 call and subtopic “Artificial Intelligence for Extended Social Interaction”. The cluster includes SONICOM, , , , and .

Three Early Career Researchers (ECR) from each of the five SECTG sister projects were invited to present. Work package leaders from each project and other ECRs were in attendance, where there was ample opportunity to ask questions, and participate in group discussions.

The day kicked off with an introduction and welcome from Coordinating partner and Principal Investigator of SONICOM, Dr Lorenzo Picinali from 51³Ô¹ÏÍø, which led into the first presentation of the day. The presentations covered a range of topics which included sharing the  (SONICOM), discussing emotional textile design through multisensory interaction design (TOUCHLESS), exploring the influence of prior acquaintance on the shared VR experience (GUEST-XR) and having a break from the screen to be led in a dance and social media workshop (CAROUSEL). 

Katharina (Kathi) Pollack, PhD student and visiting researcher at 51³Ô¹ÏÍø said, “In academia, it is essential that fellow researchers are able to communicate with each other across disciplines. I loved the idea of this event, that the early career researchers presented already useable/published outcomes of their projects and practice to debate in an interdisciplinary way. The day sparked a variety of fruitful discussions that we’d otherwise never have had!”

The day was a fantastic example of the positive impact coming together to amplify research, expand networks, and broaden horizons can have. 

The SONICOM project were invited to join celebrations of the UK's association to Horizon Europe

On December 13th, 51³Ô¹ÏÍø College brought together over 200 members of the European and global science communities to celebrate the UK’s association to Horizon Europe. SONICOM is one of the international research projects funded under the former Horizon2020 scheme that was showcased at the reception.  

The event was an opportunity for researchers, policymakers and partners and learn about some of the vital collaborations and science that EU programmes have supported at 51³Ô¹ÏÍø such as SONICOM. The project aims to leverage Artificial Intelligence to design a new generation of immersive audio technologies and techniques, specifically looking at personalisation and customisation of the audio rendering. SONICOM is led by Dr Lorenzo Picinali, Lead of the Audio Experience Design (AXP) research team with the Dyson School of Design Engineering at 51³Ô¹ÏÍø College.

Attendees taking their turn to experience binaurally spatialised audio throughout the evening

In attendance on the evening were guests including EU Commissioners, Westminster lobbyists, members of the Biochemical Society and Ambassadors from several European countries. The attendees had the opportunity to speak with researchers about their work and experience binaurally spatialised audio for themselves.

The evening was filled with excitement and optimism for a bright future ahead with Horizon Europe. Make sure to follow SONICOM on  to keep up with the latest updates on the project. 

UK Ambassador to the EU, His Excellency Mr Lindsay Croisdale-Appleby

SONICOM showcased the project to members of the public at the 51³Ô¹ÏÍø Lates: AI

On March 14th, SONICOM exhibited at the 51³Ô¹ÏÍø AI Lates. The Lates are an opportunity for the public to interact with the latest scientific developments at 51³Ô¹ÏÍø, with the theme of this Late focused on Artificial Intelligence (AI).

Across the three-hour evening, over 1000 attendees came through to check out the several exhibits and workshops on display, with 93% describing the night as a fun way to spend an evening.

SONICOM PI Dr Lorenzo Picinali was working the stall, explaining to interested members of the public alongside members of his Audio Experience Design team at 51³Ô¹ÏÍø, about research projects SONICOM and the NIHR-funded BEARS project.

The SONICOM team ran several activities with visitors to the stall which gave them a taste of what the projects are all about. One of these activities required a person to listen to four excerpts, and rate each of them on two traits; Extraversion and Agreeableness, on a scale from 0 (not agreeable/extrovert) to 10 (very agreeable/extrovert).

The purpose of this was to demonstrate how when it comes to personality traits, if you’re someone that has scored high in agreeableness, you’re likely popular and tend to make friends easily. If you score high in extroversion, this can correlate to how outgoing and warm you are perceived.

Check out how people voted below! Each colour dot (brow, white, gold and purple) correlates to each of the excerpts:

 The evening was a great success and allowed for many positive and insightful conversations to take place. Make sure to follow SONICOM on  to keep up to date with the project.

New SONICOM research has shown that machine learning approaches can be effective at spatially upsampling head-related transfer functions (HRTFs), outperforming other methods when very few measurements are taken.

New SONICOM research has shown that machine learning approaches can be effective at spatially upsampling head-related transfer functions (HRTFs), outperforming other methods when very few measurements are taken.

Published in , the research could make it easier to create high resolution HRTFs by reducing the number of necessary measurements that have to be taken and using a neural network pre-trained on existing HRTF datasets to increase the spatial resolution.

“Traditional methods [of upsampling], such as barycentric and/or spherical harmonics-based interpolation, can only rely on the data they are given to upsample, therefore in very sparse conditions performances are very much related to the number of measured positions,” said Dr Lorenzo Picinali, SONICOM’s lead PI based at 51³Ô¹ÏÍø.

“A pre-trained neural network such as the one we designed, on the other hand, can use also the data it has been trained with, resulting in significantly higher performances in very sparse conditions.”

Democratising HRTF measurement

An HRTF is a mathematical way of describing the unique way someone’s ears receive sound depending on the position of the sound source. Because everyone’s heads, ears and shoulders are different sizes and shapes, the way that sound waves are modified when they reach our eardrums is different from person to person.

Measuring your individual HRTF means it can be used to make sound coming through headphones feel more realistic with accurate distance and direction. For this reason, HRTFs are very important for creating realistic virtual reality (VR) and augmented reality (AR) environments.

However, the best way to measure high-quality HRTFs requires expensive equipment, bespoke lab environments, and the recording of thousands of measurements.

The Turret Lab at 51³Ô¹ÏÍø showcasing the equipment required to acoustically measure an HRTF.

To overcome these limitations, work has been done to see if we can ‘upsample’ low-resolution HRTFs that are made with fewer measurements into high-resolution versions – like recreating a detailed painting from just a sketch.

This new research harnesses a type of artificial intelligence (AI) called Generative Adversarial Networks (GANs), which has been successfully used in the past to upsample low-resolution images. After having trained it with data from the , when provided a low-resolution HRTF (e.g. 5-10 positions only, compared with the hundreds of the full resolution version), the GAN can create a high-resolution version.

GAN-upsampling was shown to perform better than other upsampling techniques when dealing with very limited data.

If upsampling techniques can be improved, it could make it much easier to produce high-quality HRTFs – perhaps even from simple recordings done in the comfort of your own home.

Members of the SECTG cluster met virtually for a workshop to present their work and technologies to each other in order to foster collaboration

On 23 July, members of the SECTG cluster met virtually for a workshop to present their work and technologies to each other in order to foster collaboration and enhance potential impact.

The cluster, a group of projects funded under the subtopic “Artificial Intelligence for extended social interaction”, includes five projects: SONICOM, , , , and . Representatives from each project presented an aspect of their work that was considered of interest to the others.

To kick off the workshop, GuestXR showcased their experience in haptic technologies to support social interactions in virtual and augmented reality, including devices that can provide heart rate and breathing feedback.

Members of the SONICOM team, Lorenzo Picinali and Arcadio Reyes Lecuona, then presented some of the tools and resources that have been developed within the project, including the  and the , and Giorgio Presti discussed his work on scattering delay networks.

TOUCHLESS then showcased their innovative Emosense tool, which uses AI to understand what a human is feeling in realtime using facial analysis, eye movement, body movement and speech, with possible uses including security and education.

CAROUSEL brought us to their , a virtual reality world designed to combat loneliness through shared social and physical interaction over the internet. In Plant World, visitors can dance together in a tropical, magical world inspired by the paintings of Denis Lacaux.

Finally, EXPERIENCE rounded off the presentations with their work on preference learning for optimising social behaviour in multi-agent environments.

It was wonderful to get together and discuss our projects’ respective technologies and areas of common interest, and we look forward to exploring future opportunities for collaboration.

Discover the winners of the SONICOM Listener Acoustic Personalisation Challenge.

The winners of the SONICOM  were announced at the 32nd European Signal Processing Conference () on 29 August in Lyon, France.

Sponsored by the , the LAP Challenge invited members of the auditory research community to tackle key challenges facing immersive audio technology, advance the state of the art, and contribute to the development of standardised metrics for personalised spatial audio.

The inaugural edition of the challenge concentrated on two fundamental aspects of head-related transfer functions (HRTFs): spatial sampling and interpolation. Teams were challenged to submit their solutions that address one of two tasks:

Task 1: HRTF normalisation for merging different HRTF datasets

In this task, teams were given a set of HRTFs measured from different individuals in different labs (i.e. different measurement setups, different equipment, different space, and distances, etc.) and challenged to harmonise the sets to compensate for the influences of the measurement setup.

The first prize for Task 1 was awarded to Jiale Zhao, Dingding Yao, Zelin Qiu, Chengzhong Wang, and Junfeng Li for their solution: “Normalization of Head-Related Transfer Functions Based on Neural Networks”.

Task 2: spatial upsampling for obtaining a high-spatial-resolution HRTF from a very low number of directions

In this task, teams were given a sparse set of HRTF measurements and challenged to provide a reconstructed HRTF set in high spatial resolution.

The first prize for Task 2 was awarded to Yoshiki Masuyama, Gordon Wichern, Francois G. Germain, Christopher Ick, and Jonathan Le Roux for their solution: “Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization”.

Congratulations to all the winners! A draft technical report covering the full results of the challenge is available here (a complete report and DOI will be available soon): 

Our Sponsors

A major output of the SONICOM project is now live for all to discover...

Imagine stepping into a virtual concert hall or attending a virtual meeting and hearing it exactly as you would in real life. Sounds arrive from precise directions, bounce off walls, and are subtly shaped by the unique way your head and ears interact with sound. This is the promise of spatial audio: bringing realistic, personalised sound to real‑world and virtual experiences

For years, researchers around the world have been assembling pieces of the puzzle: carefully measured head‑related transfer functions (HRTFs), spatial room impulse responses, motion‑tracking data, specialised recording setups, and bespoke software tools. These resources are scientifically valuable, but they have typically been scattered across different repositories and websites, making them hard to explore, understand, or reuse, especially for anyone outside a small expert group.

That fragmentation has now changed.

A New Home for Spatial Audio Research

The SONICOM Ecosystem is a dedicated, open, and interactive platform designed specifically for research in spatial hearing and personalised binaural audio. It brings together high‑quality data, software tools, and clear documentation in one community platform, making advanced audio research more accessible, reusable, and transparent than ever before.

The Ecosystem is a major output of the EU‑funded SONICOM project, which investigates how we experience sound socially in augmented and virtual reality, from online meetings to immersive gaming. The development of the Ecosystem was led by  at the Acoustics Research Institute of the Austrian Academy of Sciences (ÖAW).

“The SONICOM Ecosystem is a data repository for researchers in the field of spatial hearing and binaural audio. With the unique feature of visualising the data on the fly, it clearly stands out from the many general-purpose repositories. It can store tools and databases with a clear structure. That structure helps when browsing through the data, selecting the needed one for own use, and later automatic access via apps. With the integration with ÖAW Datathek, it provides persistent DOIs, ensuring sustainability and easy access world wide. The various levels of data visibility, from private to persistently published, facilitates the researchers’ typical workflow when publishing research articles accompanied by data.

The SONICOM Ecosystem provides commenting functionality, facilitating interaction between data users and providers.”

Importantly, the Ecosystem does not only store files, but it also hosts tools, software, and models for researchers to use, and shows clearly how these tools connect to the data. This means users don’t just download information and figure it out later, they can explore it, understand how it works, and easily plug it into their own projects or applications.

Data can be kept private while work is ongoing, shared openly with others, or permanently published so it can be cited in scientific papers. Each published dataset gets DOI, so it can be reliably found and used years into the future. This makes the SONICOM Ecosystem a trustworthy, long‑term resource for researchers, developers, and anyone working with spatial audio.

Why the SONICOM Ecosystem matters

Spatial audio is becoming central to many areas of technology and research, including:

• Virtual and augmented reality
• Gaming and immersive media
• Hearing aids and assistive listening devices
• Audio engineering and auditory neuroscience

But progress depends on three things working together:

• High‑quality, standardised data, such as HRTFs, spatial room impulse responses, source and sensor directivities, headphone responses, motion‑tracking data, and annotated binaural recordings
• Robust software tools and models, including measurement toolboxes, simulation frameworks, auditory models, and neural network weights used in spatial‑audio processing
• A shared scientific environment where researchers can collaborate, comment on datasets, and build directly on each other’s work

The SONICOM Ecosystem was designed specifically to bring these components together in one coherent platform.

What the Ecosystem Offers

A Curated Library of Spatial‑Audio Databases

As of today, the Ecosystem hosts 37 databases, each carefully organised so they’re easy to explore and understand. Databases consist of datasets, which in turn contain data files of specific types. All content is clearly described with simple explanations of how the data was recorded, what equipment was used, how it can be shared, and how it connects to other resources on the platform.

The databases cover a wide range of spatial‑audio data, including:

• Head‑related transfer functions (HRTFs)
• Spatial room impulse responses (SRIRs)
• Directivities (Loudspeaker, microphone, musical instruments)
• Headphone impulse responses and/or equalisation filters
• Speech and binaural recordings
• Movement and motion‑tracking data
• Supporting materials such as images, 360° images, and geometric meshes, text, tables

This structured approach makes datasets easier to browse, compare, and reuse, even for researchers new to the field.

Explore Data Before You Download It

One of the Ecosystem’s defining features is its dynamic visualisation widgets, which allow users to inspect data directly in the browser before downloading anything. Depending on the data type, users can explore:

• Three‑dimensional HRTF representations
• Spatial room impulse response plots
• Interactive 360° imagery
• Directivity “balloons” for sources and sensors
• Interactive 3D geometric meshes used in simulations or measurements

This immediate visual feedback supports quality control, faster understanding, and more informed reuse of data.

Tools That Connect Data and Practice

Beyond databases, the Ecosystem currently hosts 14 tools, including:

• Software for acoustic measurements and simulations
• Auditory and perceptual models
• Neural networks for spatial‑audio processing
• Scripts for analysis, statistics, and figure generation used in publications

What makes these tools especially useful is that they are directly linked to the relevant data on the platform. Instead of downloading files in isolation, users can see which tools work with which datasets, creating a connected and easy‑to‑navigate environment. The Ecosystem also allows external apps and software to access its data automatically, making it valuable not only for academic research but also for use in real‑world audio products.

As Piotr Majdak explains:

“The programmatic interface provides access to the data from software applications, enabling the usage of the repository data in arbitrary apps provided by audio industry.”

Designed for Contribution and Reproducibility

To help ensure high scientific quality, only verified researchers can contribute to the SONICOM Ecosystem. Contributors follow a clear, step‑by‑step process that reflects real research practice: they register using their ORCID researcher ID, describe their work using standardised metadata, define what each dataset contains, and upload their files either individually or in larger batches.

Databases can move through several visibility stages:

• Private (during creation and revision)
• Public within the Ecosystem
• DOI assigned (modifications allowed, but with a permanent link)
• Persistently published and locked for long‑term reference

This staged approach mirrors real research practice and supports transparent, reproducible science.

A Platform with Lasting Impact

The SONICOM Ecosystem was conceived not just as a project deliverable, but as a lasting infrastructure for the community.

Principle Investigator of SONICOM, from 51³Ô¹ÏÍø, highlights its broader significance:

“SONICOM includes a dedicated work package aimed at extending engagement beyond the consortium and ensuring long-term impact beyond the project’s duration. The SONICOM Ecosystem is a central outcome of this effort, supporting both active external engagement and sustained post-project impact. It stands as a major milestone not only for SONICOM, but for the wider spatial acoustics and immersive audio community. This approach sets a precedent for future projects and could evolve into a standard expectation for research initiatives in this field.”

Open source, community‑driven, and built for the long term, the SONICOM Ecosystem lays the groundwork for the next generation of personalised audio technologies.

Explore the Ecosystem at: 

Dr Piotr Majdak presenting the SONICOM Ecosystem at DAGA 2026

Visiting PhD student to 51³Ô¹ÏÍø, Manuel Marí-Altozano, provides an overview of his music experience he recently exhibited.

Manuel Emilio Marí-Altozano is a PhD student at the Department of Audiovisual Communication and Advertising of the University of Malaga. His research focuses on the construction of individualised experiences for music through listeners’ exploration. He uses virtual reality and spatial audio to build music experiences which allow the user to explore a 3D musical environment designed for a specific room. This project introduces the concept of the open world, widely used in video games, to the realm of music and the performing arts.

Kathi Pollack explains how the shape of our ears helps us understand where sounds come from in everyday life

In everyday indoor environments, we can locate sound sources easily, even with our eyes closed. One reason for that is room acoustics, and the way sound is reflected and absorbed by the materials in the room. Take the room away, and we retain this ability. Why? Because of the way the human body is shaped.

Sound is absorbed and reflected by our bodies. When sound arrives at our eardrums, its properties have changed – some parts are reduced, and others are enhanced– for each position a sound can come from. This position-dependent spectrum is known as a head-related transfer function (HRTF). It acts as a personal acoustic filter, shaped by our individual anatomy.

Imagine a drone starts hovering in front of you, and you have your eyes closed. The drone flies straight up above you, and lands on the ground behind you. For all the drone’s positions, there were no time or level differences between the ears. The reason we can still easily locate the drone is that our ears have a particular shape. Not only is the shape of the outer ears unique, but our ears are individual, similar to a fingerprint (yes, even left and right are different from one another!). During our lives, we have learnt to map the filtering effect of our ears to specific locations around us.

To see this in action, check out popular engineer and online educator, Mark Rober’s short demonstration of how our ear structures are important to the way we locate sound:

 

 

For sound source positions to the left and right of us, there are time and level differences that occur between our ears. However, some sounds, even if positioned in slightly different positions, would give us the same or no difference between our ears at all; for example, a sound that is positioned right in front of us or behind us, or above or below us. 

All the cues we use to locate sound are part of an individual HRTF. The ability to locate vanishes when we listen to sound over headphones, because the sound is not filtered by our anatomy, and thus, we perceive sound within our head. However, when we mix the headphone signal with an individual HRTF, we can trick the brain into perceiving the sound outside our head. As a result, we can feel more immersed in the movie, music, conversation, or anything that we’re listening to. This is one of the objectives of the SONICOM project. 

The challenge of individual HRTFs 

Because of our complex and unique ear shapes, it is not easy to provide individual HRTFs to a large number of people. HRTFs can be measured acoustically or calculated from a high-resolution scan of the head and ears. Both methods are often cumbersome and require a specific room and special, often very expensive, equipment. 

 is currently one of the largest datasets available of both acoustically measured HRTFs and high-resolution scans of the torso, head, and ears of over 300 participants. Within the project, we have followed several approaches to model and approximate individual ear shapes, so that in the future, we don’t have to acquire a high-resolution scan or measurement for every person using special equipment. 

The Turret Lab at 51³Ô¹ÏÍø, where researchers are collecting HRTFs as part of the SONICOM project

Broader applications

Individual HRTFs have applications beyond entertainment. By tailoring headphone playback to a listener’s unique hearing pattern, individual HRTFs can reduce listening fatigue. An individual HRTF can also help the brain to identify dialogue in a noisy environment. 

Additionally, research into HRTFs and spatial hearing is crucial for developing and improving hearing-assistive devices, such as hearing aids or cochlear implants. One promising application of using HRTFs, in an assistive way, is auditory navigation for visually impaired people. SONICOM partner, , has developed a mobile application, , to guide a person to a specific point using virtual sound sources played through headphones. This becomes incredibly helpful when a visually impaired person wants to locate something, such as a door in a building. 

The interdisciplinary approach of SONICOM is a key strength. Researchers from a wide range of disciplines and backgrounds are collaborating to advance both fundamental and applied hearing science. Through SONICOM, spatial hearing research is not only being pushed forward but also applied to practical applications that offer real-world benefits. 

Guest Author

Katharina (Kathi) Pollack is a PhD student at the Acoustics Research Institute, part of the Austrian Academy of Sciences in Vienna. Her research focuses on making personalised 3D audio more accessible to everyone. Kathi has developed a parametric model that can create virtual versions of different human ears. This means people no longer need special equipment or detailed scans to enjoy realistic and customised sound experiences through headphones.

'The Immersive Sound Lab' experience launched at the Great Exhibition Road Festival

On 7–8th June, SONICOM exhibited at the Great Exhibition Road Festival (GERF), an annual celebration of science and the arts that draws over 50,000 visitors to South Kensington across one vibrant weekend.

GERF attracts over 50,000 visitors over the weekend, featuring several major institutions on and around Exhibition Road, including the Natural History Museum, the Victoria & Albert Museum, the Science Museum, the Royal Albert Hall, and the Royal College of Art.

As part of the festival, SONICOM partners: the Dyson School of Design Engineering at 51³Ô¹ÏÍø and creative technology studio Reactify, launched an exciting new interactive experience for the public.

‘The Immersive Sound Lab’ is a cutting-edge augmented reality (AR) experience exploring how sound interacts with the physical world. For 10 minutes, visitors were active participants in a specially prepared space where physical and virtual elements merged. Their task? To see if they could trust their ears, and identify if the sound was coming from the objects around them or if it was all an auditory trick.

By taking part, visitors contributed to ongoing research for SONICOM, helping researchers and designers better understand how humans perceive spatial audio. Their insights will inform the next generation of immersive environments, with potential applications in entertainment, education and beyond.

Watch the video below to get a sneak peek of the weekend 👇

The next time the consortium all meet in-person will be for the project's final meeting

The SONICOM consortium came together in Milan, Italy, for the project’s 4th in-person annual meeting, held in a beautiful and sunny setting.

The two-and-a-half-day meeting was jam-packed with work package updates, lively discussions and opportunities to look ahead to the final 12 months of the project.

The meeting unofficially kicked off on Tuesday evening, as most of the consortium met for a relaxed aperitivo in the vibrant district of Porta Sempione. The following morning, the meeting began nice and early at the University of Milan, where obligatory espresso and croissants were enjoyed by all.

Once sufficiently caffeinated, SONICOM PI, Dr Lorenzo Picinali, welcomed everyone and gave an overview of what the next few days would entail. The morning opened with a general update from the Project Management team, swiftly followed by an overview from the Immersion work package. A delicious lunch provided a welcome break before the afternoon resumed with presentations and demonstrations from the Interaction and Integration work packages.

The meeting concluded at around 6 pm, at which point it was time to unwind and enjoy an Italian feast together at the consortium dinner.

Day two brought everyone back to the University (and back to the espresso) for a deep dive into the work of the Experience team, who presented new developments and upcoming activities. The Communication and Dissemination team then presented on the achievements from the past 12 months and collaborated with the consortium on activity ideas until the end of the project. The Beyond work package presented and led an engaging discussion on the SONICOM Ecosystem. After another wonderful lunch, the consortium regrouped to discuss opportunities beyond SONICOM.

The final (half) day focused on follow-up proposals and testing various demonstrations, which included Reactify’s experience from the Great Exhibition Road Festival, indoor navigation by Dreamwaves, and spatial audio in teleconferencing.

The meeting concluded on a high note and was deemed a great success. A heartfelt thank you to the Milan team for their warm hospitality and for hosting such a well-organised and productive gathering.

The consortium is energised as they head into the final year of SONICOM and already look forward to reconvening for the final meeting next year.

Discover more about how the SONICOM tool BeRTA can enhance virtual reality applications.

Close your eyes and listen to the world around you for a moment. Whether it’s birdsong, cars, the TV, or coworkers typing away at their desks, you’ll experience a variety of sounds that have depth, distance and direction.

Simulating this kind of 3D soundscape through headphones is known as spatial audio. As this technology becomes more important in fields like virtual reality, interactive media, and audio perception research, there’s a growing need for tools that are not only powerful and accurate, but also accessible and flexible.

That’s where the Binaural Rendering Toolbox (BRT) comes in. Developed by researchers from the SONICOM project at the University of Málaga and 51³Ô¹ÏÍø, BRT is a modular, open-source library designed to simulate realistic spatial sound in real time. Built with researchers and developers in mind, it allows precise control over how sound sources behave, how spaces respond acoustically, and how listeners perceive those environments.

On top of BRT, we’ve built BeRTA, a standalone application that brings this technology to life in a user-friendly way. BeRTA connects seamlessly with popular graphics engines like Unity, making it easy to create immersive auditory scenes for experiments, virtual environments, or artistic projects. Whether you’re studying human perception or designing rich interactive soundscapes, BRT and BeRTA provide the tools to do it with precision and creativity.

A powerful, flexible tool

BeRTA is a highly configurable application built on a modular architecture, allowing users to design a wide range of virtual sound scenarios. It handles all audio processing and spatialisation through the Binaural Rendering Toolbox (BRT), which builds upon algorithms originally developed in the 3D Tune-In Toolkit, expanding their functionality within a more flexible and research-oriented framework.

BeRTA integrates additional open-source components for tasks such as file handling, graphical user interface rendering, and communication via Open Sound Control (OSC). The entire system can be controlled remotely using OSC commands, while the GUI primarily serves as a visual aid for configuration and monitoring. A startup configuration file allows users to define default parameters and customize auditory scenes to meet specific experimental or creative needs.

BeRTA supports multi-listener setups, simulates both omnidirectional and directional sound sources, and generates binaural output using HRTF and BRIR models. It recreates realistic room acoustics through techniques like Scattering Delay Networks, and includes binaural filters for perceptual compensation and fine-tuning.

This versatility makes BeRTA a robust solution for advanced auditory research and interactive spatial sound design.

More yet to come

BeRTA and the BRT Library are under active development, with upcoming features including advanced hybrid reverberation models, support for ambisonic sound sources, and tools for simulating hearing impairments.

As a versatile, open-source solution for spatial audio, BeRTA is designed to meet the needs of researchers and developers working in virtual reality, auditory perception, and immersive media. Its modular architecture, real-time rendering capabilities, and flexibility enable the creation of rich, realistic soundscapes that go far beyond what conventional audio tools offer.

You can explore the documentation, access the source code, and start using BeRTA today via our official GitHub repository: 

Guest Authors

Maria Cuevas Rodriguez earned her M.Sc. degree in telecommunications engineering in 2011, M.Sc degree in mechatronic in 2013 and PhD degree in 2022 at the University of Malaga (Spain). In 2009 she moved for a year to Tatung University (Taiwan). In 2011, she started her career in the department of Electronic Technology of University of Malaga. Since then, she is working as a researcher in different European Projects (ManuVar, Use-it-Wisely, Pluggy, 3D Tune-In and currently in SONICOM), in the fields of Robotics and Virtual Reality, 3D binaural audio, psychoacoustics and 3D interaction. She pursed her Ph.D. about “3D Binaural Spatialisation for Virtual Reality and Psychoacoustics”. During her PhD she has accomplished two internships, at the 51³Ô¹ÏÍø for 3 months and at the audio team of Meta (formerly Facebook Reality Lab) for 6 months. She is now actively working on the development of the Binaural Rendering Toolbox (BRT), after having contributed significantly to its initial development, the 3D Tune-In Toolkit.

Daniel Gonzalez-Toledo is a researcher at the University of Malaga and a member of the DIANA research group. He obtained his PhD in 2024, specializing in advanced 3D interaction techniques with reduced degrees of freedom. He holds a BSc+MSc in Telecommunication Engineering from the University of Malaga. He began his professional career at IBM as a testing engineer. Since 2013, he has been involved in multiple European and Spanish research projects in the fields of Virtual Reality, focusing on 3D interaction, spatial audio, virtual reality for cultural heritage, and haptic environments. As part of his research, he contributed to the development of the successful 3D Tune-In Toolkit and is now actively involved in its reedition, the Binaural Rendering Toolbox (BRT). He has also completed research stays at 51³Ô¹ÏÍø and the National Technical University of Athens. His work includes contributions to various studies on spatial audio and 3d interaction. Currently, he is part of the Horizon 2020 project SONICOM.

Arcadio Reyes-Lecuona received a BSc+MSc degree in Telecommunication Engineering in 1995 from the Univ. of Málaga, a BSc+MSc degree in Psychology in 2017 and a PhD degree in 2001 from the same university. He is head of the Advanced HCI and VR Reality Research group (DIANA) at the Univ. of Málaga. He is a member of the steering committee of the Spanish HCI association (AIPO), member of EuroXR, the European association for Extended Reality and member of the AABBA group, an intellectual open group of scientists collaborating on development and applications of models of human spatial hearing. His specific research interests include 3D Binaural Audio, 3D interaction and evaluation of interaction techniques for virtual environments. He is the Principal Investigator of the University of Malaga team in the SONICOM project.

Discover more about SONICOM's partner Dreamwaves and their 3D audio navigation journey.

Finding your way through town, even if it’s your own, can be a hassle.

We’ve all been there: you leave the subway station, already a bit late for your dentist appointment, and you turn and turn your phone, trying to understand in which direction to take your first steps. You try different options, waiting for the arrow to start moving in the right direction, perhaps dancing a little involuntary conga on the platform. All very amusing, if it weren’t for that aching tooth of yours. If only the dentist could lean out of the window and shout your name – “Hey, over here! Come this way!” – that would surely help.

But why is this experience so common? It turns out that reading a map is not intuitive, but a skill that needs to be learned. Some of us do it better, others worse. GPS systems help but don’t always work seamlessly. Sometimes they can’t immediately recognize our exact location, or they don’t show it clearly enough to scale. To complicate things further, when we are nervous, our cognitive system feels overloaded and tends to work worse: easy tasks suddenly seem harder.

An intuitive signal – something that your brain doesn’t need to actively process to make sense of it – is the ideal solution for navigation. Just like the dentist shouting your name out of the window.

This notion is at the very core of . We aim to challenge the way we navigate by guiding you through a series of 3D sound cues, as opposed to a visual map. Imagine making your way to your destination and, instead of constantly checking your location on your phone, simply following a series of sounds that guide you along the way. You hear them through your headphones as if they’re coming from where you need to go next. All loud and clear. No more hesitant congas. You hear it, you walk to it. Point after point until you reach your destination.

 

 

This accuracy, plus the unique audio experience of our app, gave us an idea: did our work have the potential to help blind and visually impaired people navigate the world? The possibility of empowering people by offering them more freedom of movement was thrilling. Thus, together with visually impaired users, we are improving our app to adapt it to their specific challenges and needs.

Our involvement in SONICOM allows us to refine our 3D audio navigation technology by integrating cutting-edge spatial audio research and AI-driven sound processing. This collaboration helps us enhance accuracy, improve user experience, and develop more intuitive and inclusive navigation solutions.

This is our 3D audio navigation journey. Visit  to find out more about how we intend to make navigation easier for all, getting you to your destination using your ears (and your phone).

Guest Authors

Xaviera Torres manages communications for Dreamwaves, bringing her expertise as a science communicator, author, and podcaster to make complex topics accessible to all audiences. A biologist with a PhD and a Master’s in the History of Medicine, she specializes in translating scientific and technological advancements into engaging, understandable content.

Hugo Furtado, PhD, is the founder and CEO of Dreamwaves, a startup pioneering spatial audio navigation technology to improve accessibility. With a background in engineering, programming, VR, and AR, he has worked at CERN, in the IT industry, and as a postdoctoral researcher at the Medical University of Vienna. Hugo specializes in innovation management, translating scientific advancements into impactful products.

SONICOM researchers recently presented an extended version of the SONICOM Head-Related Transfer Function (HRTF) dataset at Forum Acusticum - Euronoise 2025

The external part of the human ear – the pinna – has a unique and intricate shape that varies significantly from person to person, making it difficult to accurately determine and recreate its individual properties. 

 introduces BezierPPM – a new parametric model of the pinna based on cubic Bézier curves and concave deformations. This parametric pinna model (PPM) represents the biological structure of the human pinna and enables its parametric modelling in many degrees of freedom. 

“In contrast to other PPMs, BezierPPM is inspired by the biological structure of the human pinna aiming at providing parameters directly related to the complex pinna geometry,” says Felix Perfler, PhD student at the Acoustics Research Institut of the Austrian Academy of Sciences and first author of the publication. “Our analysis shows that BezierPPM can accurately represent human pinnae, meaning that it has high potential for a range of applications, including biometric identification, the design of ear prostheses, and the personalisation of spatial hearing.” 

Capturing complex geometry 

Accurately capturing the complex morphology of the human pinna can prove challenging. Direct optical capture of pinna geometry potentially offers the required accuracy, but requires special equipment such as laser scanners and the result must be post-processed to remove artifacts resulting from the scanning process. This post-processing is usually performed manually and requires a considerable time and expertise. 

By representing the pinna geometry parametrically, this process can be simplified: parameters of parametric pinna models can be modified such that the modelled mesh matches the target pinna mesh, a process referred to as registration. 

BezierPPM offers a PPM that uses parameters closely linked to human ear structures, representing the ear geometry as cubic Beziér curves and including local modifiers of predefined concave areas. 

BezierPPM was evaluated by manually registering its parameters to 20 ears selected from various databases of digitized ears. Analysis shows that BezierPPM was capable of representing human pinnae very well, and that most of the inaccuracies were located on the back side of the ear, an area having a rather low relevance for most target applications. Further, BezierPPM was tested in a machine-learning setting to show that it can be applied in future AI-based algorithms. 

While the BezierPPM is ready to be used, further evaluation will be required for specific use cases such as the customization of ear prostheses or HRTF personalization. In particular, for applications in acoustics and binaural audio, evaluations in the psychoacoustic domain in terms of various spatial-hearing performance metrics when listening with numerically calculated HRTFs obtained from BezierPPM-based meshes is necessary. 

Read the  to find out more, or access BezierPPM directly on .

New SONICOM research introduces BezierPPM – a new parametric model of the pinna based on cubic Bézier curves and concave deformations.

Understanding how humans perceive sound is a complex challenge, and scientists historically rely on computational models to describe the perceptual mechanisms behind listeners’ behaviour and their neural underpinning.

To address the need for standardised and reproducible methods,  introduces FrAMBI, a MATLAB/Octave framework that applies Bayesian inference to model auditory perception and behaviour.

FrAMBI provides researchers with a statistical framework for implementing and evaluating auditory models, ensuring consistency across studies. By structuring models around the perception-action cycle, FrAMBI simulates how listeners dynamically engage with sound sources, enabling hypothesis testing and computational statistical analysis.

“We created FrAMBI to provide the auditory community with a common statistical methodology to generate empirical evidence through auditory models,” says Dr Roberto Barumerli, the publication’s lead author based at the Austrian Academy of Sciences and the University of Verona. “We hope that the toolbox will foster reproducible research in psychoacoustics and auditory neuroscience.”

The new publication presents FrAMBI’s theoretical foundation and demonstrates its capabilities through examples of increasing complexity in sound source localisation tasks:

• Implementation of a basic static scenario;
• Evaluation of dynamic scenarios with a moving sound source;
• Comparison of model variants testing neural mechanisms for vertical localisation.

FrAMBI is integrated into the widely used and SONICOM-supported , allowing it to be maintained long-term and expanded accordingly.

To learn more about FrAMBI and its possible applications, read the full publication: 

To experience FrAMBI firsthand—, load it in MATLAB, and run exp_barumerli2024(‘basic’) to explore its capabilities.

New SONICOM research surveys machine learning-based approaches for HRTF personalisation in the literature.

Spatial audio relies on head-related transfer functions (HRTFs) to simulate how sound interacts with the human body in order to create realistic audio experiences. Since HRTFs are unique to each person, using non-personalised HRTFs can lead to audio experiences feeling poorer and less immersive.

Machine learning-approaches have shown that they can be more efficient and accurate than traditional methods when it comes to HRTF personalisation. However, these methods require careful training and validation to achieve good and unbiased performances. Despite this, there is as yet no standard approach to evaluating these methods.

 surveys machine learning-based approaches for HRTF personalisation in the literature, organising them according to the processing steps involved in the machine learning workflow.

In addition to categorising the works of the existing literature, this survey discusses their achievements, identifies their limitations, and outlines aspects that require further investigation at the crossroads of research communities in acoustics, audio signal processing, and machine learning.

“Previous surveys have covered various aspects of HRTF personalization, but none have focused specifically on machine learning.” says Dr Davide Fantini, lead author of the paper based at the University of Milan. “We hope that this survey will provide other researchers with a clear overview of the state of the art and encourage future standardization actions in the field of HRTF individualisation based on machine learning.”

Results

The analysis revealed the prevalent approaches for each step of the machine learning workflow, which include:

1. CIPIC – one of the earliest HRTF datasets – as the dataset;
2. Anthropometry as input;
3. HRTF magnitude as output;
4. Possibly principal component analysis (PCA) for HRTF preprocessing;
5. Multivariate linear regression (MLR) or neural networks as machine learning model, and;
6. Spectral distortion for evaluation.

Subsequently, the research discussed the main gaps existing in the literature which could comprise topics of future studies, i.e.

1. The limitations of anthropometry-based methods;
2. The reported performances, which are still inferior to the individual HRTFs;
3. The scarce applications of recent machine learning developments, including explainable AI;
4. The lack of a standardized evaluation protocol, and;
5. The infrequent investigation of perceptual metrics, especially in the context of ecologically-valid experimental settings, which encompass multiple aspects of spatial audio beyond HRTF individualization.

Read the full publication: 

The SONICOM team exhibited at the event in what was the project's last engagement activity of 2024

On 5 December, SONICOM participated in the 51³Ô¹ÏÍø Lates: Senses evening. SONICOM isn’t a stranger to the Lates series, having taken part in a few earlier iterations. However, given that the theme of this particular event focused on the senses, it was an unmissable opportunity to be involved.

On the night, SONICOM was positioned alongside friends from  another project involving SONICOM PI, Prof. Lorenzo Picinali. This provided a great opportunity to show how the research from the SONICOM project underpins real-world implications showcased in BEARS as they aim to maximise hearing in young people with bilateral cochlear implants.

The SONICOM team ready to welcome visitors to the stall

Dr Katarina Poole led SONICOM’s efforts for the evening, coordinating various hands-on activities and demonstrations designed to immerse attendees in the science of sound, perception, and immersive audio technologies. Visitors also had the opportunity to book their space on a tour of the (though tours did fill up very quickly!) to experience the thunder dome where many SONICOM experiments take place.

The event was a great success with over 600 participants throughout the night with 93% of visitors describing the event as a fun way to spend an evening and 96% saying they would attend another Lates event!

The 51³Ô¹ÏÍø Lates: Senses evening was SONICOM’s final public engagement activity for 2024, rounding out a year filled with impactful outreach initiatives, stay tuned for even more ambitious and engaging activities planned for 2025!

Members of the SONICOM team showcasing the research with visitors

Visitors had the opportunity to experience a range of SONICOM experiments for themselves

Spatial Audio Metrics (SAM) is an open-source software platform to help researchers and audio enthusiasts compare and visualise HRTFs

As spatial audio technology continues to advance, accurate and reliable evaluation methods are essential. 

Introducing  – an open-source software platform to help researchers and audio enthusiasts compare and visualise Head Related Transfer Functions (HRTFs) with new features being continually added.  

Created by SONICOM researcher Dr Katarina Poole at 51³Ô¹ÏÍø, SAM was made in order to provide open-source tools that allow easier analysis of data within the spatial audio domain. 

“Spatial audio plays a key role in creating fully immersive and captivating experiences in augmented and virtual reality (AR/VR),” says Dr Poole. “The SAM toolbox is designed to enable researchers to fine-tune how accurately spatial audio replicates real-world listening in order to make it as immersive as possible.” 

SAM offers the ability to evaluate various aspects of spatial audio, including: 

• Numerical precision: Examine the auditory cues embedded in HRTFs, such as frequency, timing, and level differences. This analysis helps evaluate how effectively different HRTFs perform relative to one another 
• Perceptual fidelity: Gain insights through behavioural analysis into the perceptual impact of HRTFs by identifying areas where they succeed or fall short in delivering accurate spatial audio experiences
• Visualisation: Leverage intuitive visual tools to pinpoint differences between HRTFs, whether they manifest in the time domain, frequency domain, or both.

By utilizing SAM, researchers, developers, and audio engineers can gain valuable insights into the spatial performance of their audio content and make informed decisions to enhance the listening experience.

Visit the SAM repository .

SAM allows you to quickly visualise spectral differences between HRTFs at each individual location

The SONICOM HRTF Dataset is being used as part of an ‘Individual Head-Related Transfer Functions (HRTFs)’ competition organised by Huawei.

The  is being used as the main data input for training the AI model used in an  organised by Huawei.

The objective of the competition is to develop a method to synthesize/reconstruct HRTFs based on pictures of the person’s ears, and the SONICOM HRTF dataset of 3D meshes, 2D pinna images and corresponding acoustically measured HRTFs is provided as the input for model training.

“It’s great to see our open-source SONICOM HRTF dataset being utilised in this way,” says Prof Lorenzo Picinali, SONICOM’s Coordinator based at 51³Ô¹ÏÍø. “We look forward to seeing the innovative solutions the teams who take part end up producing.”

The SONICOM HRTF Dataset includes different types of data measured from 200 subjects, including HRTFs, HpTFs, depth pictures and 3D models of subjects’ ears, heads and torsos, with further measurements ongoing. All of this data is open-source and .

More details about how the dataset has been measured can be found in our .

Take part in the competition

Registration for the competition closes on 30 November 2024, with solutions due by 4 December. Participants must be studying a BSc, MSc, or PhD in engineering, maths, computer science or another related field at a higher education institution in Europe or the UK in order to be eligible.

Top performing teams, up to 50 people, will then be invited to the final in Germany pitch their innovative solution in front of a Huawei Jury Panel for a chance to win prizes ranging from cash and travel and internship opportunities.

Find out more about the competition on the .

The project hosted ECRs from across Europe for a full day workshop to start preparing them for funding opportunities

On the 20th of May 2026, the SONICOM project coordinated a dedicated workshop for Early Career Researchers (ECRs), designed to deepen their understanding of what is required when writing a grant application and how the process of securing funding works across the UK and Europe. The workshop brought together 14 participants from 7 institutions, providing a great opportunity for networking and collaboration.

Researchers began arriving from 9:30am, and the day opened with a welcome from , Principal Investigator (PI) of SONICOM. He began by expressing his motivation behind organising the workshop: ‘It was easier for me to compete and find funding 20 years ago; today, not so much. So, I want to do something to help you, the newer generation coming through, to thrive in this more intense, competitive funding landscape. Hence why we are here together.’ 

Prof Lorenzo Picinali, PI of SONICOM, welcoming ECRs to the workshop

Prof. Picinali continued by providing an overview of his career trajectory, deliberately highlighting both failures and successes, reminding participants that this up-and-down cycle is normal in any researcher’s career.

Following on, Branwen Hide, Research Development Manager at 51³Ô¹ÏÍø, who is the primary advisor for 51³Ô¹ÏÍø’s academic research community interested in collaborative funding opportunities with the European Commission, gave an overview of how European funding works.  Branwen spoke through key funding opportunities in the European Commission’s Horizon Europe scheme, places to connect, and where to get the right information from. 

The session prompted a highly engaged response from the researchers, with questions and discussion flowing freely. So much so that the day was already running behind schedule, which resulted in a swift break for coffee and pastries before the group reconvened to continue asking questions. 

Next, , Head of Impact Management with the Research Impact Management Office at 51³Ô¹ÏÍø and Project Manager of SONICOM, guided participants through the critical groundwork required before starting to write a proposal. Siobhan covered understanding the expectations, scope, and eligibility of a funding call; how to maximise alignment with the call, objectives and the design of your proposal; and navigating the distinction between what is mandatory and what is encouraged. 

Ivana Balkan, Innovation Manager also within the Research Impact Management Office, built on Siobhan’s presentation by focusing again on what to include in a proposal, this time honing in on writing for impact. The session explored how to construct a clear and persuasive case for why research deserves funding, framing research in terms of broader societal, economic, and scientific impact, and common pitfalls in writing for impact and how to avoid them.

A well-earned lunch break allowed participants to decompress, reconnect with familiar colleagues, and forge new connections. 

The afternoon shifted from theory to practice, with participants being tasked to work in groups to craft an idea for a proposal and to try to find a call that aligned with their idea. Each group had to consider an original research idea suitable for a grant proposal, identify a relevant funding call that aligned with their concept, and craft the early foundations of a proposal, applying the morning’s learning in real time. Following a refreshment break, the workshop reconvened for its final session: the group presentations. 

Each team presented their proposed research idea and identified a funding call to the room, before receiving feedback from Prof Picinali, Siobhan and Ivana from the morning’s session, and Communications Manager for SONICOM, Alexandra Rayner. The four offered constructive, personalised feedback on each proposal, drawing on the themes explored throughout the day, from alignment with funding calls and eligibility considerations to impact framing and narrative clarity. 

A team of ECRs presenting their proposal for a funding call

Prof Picinali brought the day to a close, reflecting on the day’s discussions and encouraging participants to carry the lessons learned into their own funding journeys. The wrap-up reinforced the workshop’s central message, that navigating the funding landscape is a skill that can be learned, and that building the right networks, asking questions, and writing with clarity and purpose are all within reach for the next generation of researchers. 

One ECR, Dr Rapolas Daugintis, reflected on his key takeaway of the workshop: ‘Research funding is extremely competitive and requires a lot of perseverance. However, one can improve their chances by understanding the rules of the bidding game.’

SONICOM’s Project Manager (and co-organiser of the ECR workshop) reflected on the day:

“Workshops like this one are crucially important for any researcher. Whether they be starting out in their career or are well-seasoned, no funding scheme should be considered as stagnant. It fluctuates with the moving tides of a multitude of variables from international policy and government elections to local funding possibilities and institutional priorities.

Really understanding the landscape in which you are applying for funding, must be navigated with a critical eye and with creative ambition; it is an art form. This wonderful lot of passionate researchers are early in their careers, so it was wonderful to impart some of my knowledge and navigational tools so that they can not only catapult their career prospects, employability, and upscale their research value, but also turn their science into art.”

The day concluded on a social note, with participants and speakers gathering for casual drinks at Beit Hall, providing a relaxed and informal setting to continue conversations sparked throughout a very successful day. 

Siobhan Markus, Head of Impact Management in the Research Impact Management Office and Dr Rapolas Daugintis, University of Surrey, during the workshop