Saturday, November 19, 2016

Seeing Is Believing - Mixing Digital Sights and Sounds

Thanks to a new EU-funded project, our listening experience of electronic music is about to become a lot more harmonious. With the help of IXMI’s visuals, both the musician and the audience can enjoy an all-round superior performance.

Today, when we think of the digital music business, we mainly think of music created by digital techniques rather than the art form of electronic music. However, the recent development of digital musical instruments has given musicians a great opportunity to broaden their creative horizons and become more experimental and adventurous in their artistic creations.

Credit: © ricardoferrando -

Despite these developments, audiences seem unable to perceive how digital musicians control the sounds they create. This affects the listening experience compared to acoustic musical instruments or traditional live performances, for example.

Sound approach

The IXMI project set out to explore ways to address audience perception of how artists interact with digital systems.

“Spectators can’t rely on physical properties of [digital] instruments to deduce the sounds they produce,” notes IXMI’s coordinator Florent Berthaut. This is in sharp contrast to acoustic musical instruments with which the audience is likely to be more familiar, having either played or seen them played before.

The project successfully addressed these issues by using augmented reality display software that amplifies the musician’s gestures with virtual extensions of the digital instruments (demonstrated in this video). The team also sought to display the actual sound processes at work, indicating to those listening which part of the music is manipulated by the musician’s gestures and how.

Research into the ‘liveness’ of digital music has suggested that a change in the audience’s perception of how much control the musician has over the music may significantly affect their enjoyment and emotional engagement with the artistic performance.

The IXMI project developed a mixed-reality display that enables spectators to explore and perceive various visual augmentations in the performance by using their own reflections in a semi-transparent mirror – the hardware and software is freely available online under an open-source licence.

Mirror image

These findings, including the software and hardware components, not only benefit users of digital music – musicians and listeners – but also contribute to other areas of research. For example, the IXMI mirror is being used in museums to enhance visitors’ experience of exhibited objects. The project’s work has also been used in a PhD student’s research on the physiological measurement of ‘liveness’.

The successful project, funded under the Marie Curie Intra-European Fellowship, also gave a significant boost to Berthaut’s own career, helping him attain a permanent position as an Assistant Professor (MaĆ®tre de Conferences) at the University of Lille.

Above all, IXMI’s outputs will have a great impact on the perception of electronic music performances by the general public. According to Berthaut, it has taken the first steps towards transforming digital music into an art form and cultural experience that could attract a much broader audience in future, particularly in live performances.

“It will improve the familiarity of spectators with these instruments and help them appreciate electronic music performances, in particular the engagement and virtuosity of electronic musicians when they perform,” explains Berthaut.

The work also opens the door to potential future improvements in all interaction between users, live performances, exhibited objects or other traditional ‘physical’ experiences by offering novel digital methods to better these.

Contacts and sources: 
EC Research and Innovation

No comments:

Post a Comment