This journal article summarizes the most relevant results we found throughout my master thesis research – namely, the results related to popular western music. However, in this thesis we also describe the first attempt of remixing orchestral music for improving CI users classical music experience. Although the results for orchestral music are not conclusive, they provide nice intuition for designing future experiments and might be valuable for researchers who are interested in that topic.

Abstract – Music perception remains rather poor for many Cochlear Implant (CI) users due to their deficient pitch perception. However, comprehensible vocals and simple music structures are well perceived by many CI users. In previous studies researchers re-mixed songs to make music more enjoyable for them, favoring the preferred music elements (vocals or beat) attenuating the others. However, mixing music requires the individually recorded tracks (multitracks) which are usually not accessible. To overcome this limitation, Source Separation (SS) techniques are proposed to estimate the multitracks. These estimated multitracks are further re-mixed to create more pleasant music for CI users. However, SS may introduce undesirable audible distortions and artifacts. Experiments conducted with CI users (N=9) and normal hearing listeners (N=9) show that CI users can have different mixing preferences than normal hearing listeners. Moreover, it is shown that CI users’ mixing preferences are user dependent. It is also shown that SS methods can be successfully used to create preferred re-mixes although distortions and artifacts are present. Finally, CI users’ preferences are used to propose a benchmark that defines the maximum acceptable levels of SS distortion and artifacts for two different mixes proposed by CI users.

Links:

• Journal of the Acoustical Society of America reference
• Paper
• Code for the mixing console – a web app multitrack player.
• Code for the DRNN source separation algorithm.
• Code for the NMF source separation algorithm.
• Master thesis

References:

• Jordi Pons, Jordi Janer, Thilo Rode, & Waldo Nogueira. Remixing music using source separation algorithms to improve the musical experience of cochlear implant users. The Journal of the Acoustical Society of America, 140(6), 4338-4349. 2016.
• Jordi Pons. Music remixing using source separation to improve cochlear implant users music perception. Master Thesis, 2015. Universitat Pompeu Fabra.

Acknowledgments: We would like to thank Wim Buyens for sharing the audio excerpts used for this study. Special thanks to the CI users from the German Hearing Center of the Medical University of Hannover and to the NH listeners that volunteered to participate in these extensive experiments. Also thanks to the anonymous reviewers for their suggestions and comments. This work was supported by the DFG Cluster of Excellence EXC 1077/1 “Hearing4all.”