I was invited to give a talk to the Deep Learning for Speech and Language Winter Seminar @ UPC, Barcelona. Since UPC is the university where I did my undergaduate sudies, it was a great pleasure to give an introductory talk about how our community is using deep learning for approaching music technology problems.
Download the slides!
Overall, the talk was centered in reviewing the state-of-the-art (1988-2016) in deep learning for music data processing in order to boost some discussion about current trends. Several key papers were chronologically listed and briefly described: pioneer papers using MLP , RNNs , LSTMs  and CNNs  for music data processing; and pioner papers using symbolic data , spectrograms  and waveforms  – among others.
Given that several relevant researchers in our field were in Barcelona for being part of the jury of Ajay‘s and Sankalp‘s PhD thesis defense, the MTG hosted a very interesting seminar. Among other topics, the potential impact of deep learning in our field was discussed and almost everyone agreed that it seems that end-to-end learning approaches are not successful because no large-scale (annotated) music collections are available for research benchmarking. And indeed, most successful deep learning approaches use those models as mere feature extractors or as hierarchical classifiers build on top of hand-crafted features.
A brief review of the state-of-the-art in music informatics research (MIR) and deep learning reveals that such models achieved competitive results in a relatively short amount of time – most relevant papers were published during the last 5 years. Many researchers successfully used deep learning for several tasks: onset detection, genre classification, chord estimation, auto-tagging or source separation. Even some researchers declare that is the time for a paradigm shift: from hand-crafted features and shallow classifiers to deep processing models. In fact, in the past, introducing machine learning for global modeling (ie. classification) resulted in a significant state-of-the-art advance – no one doubts about that. And now, some researchers think that another advance could be done by using data-driven feature extractors instead of hand-crafted features – meaning that these researchers propose to fully substitute the current pipeline by machine learning. However, deep learning for MIR is still in its early ages. Current systems are based on solutions proposed in the computer vision, natural language or speech research fields. Therefore, now it is time to understand and adapt these for the music case.