A Framework for Synthesis of Musical Training Examples for Polyphonic Instrument Recognition

Abstract

Music information retrieval (MIR), an interdisciplinary field involving the classifying or detection of structure in music, is essential for processing, indexing, querying and making recommendations from the vast amount of musical data available on the web and in audio library collections. Deep neural networks have yielded state-of-the-art results in several MIR tasks, but are often limited by their reliance on the availability of large amounts of annotated training data. Thus, applying deep learning in MIR may prove difficult when applied to music databases with limited amounts of labelling. This thesis addresses the question of whether algorithmically compositions generated from a specification of instruments and note events may serve as a viable alternative to real labeled music recordings for use as training data in MIR classification tasks. We propose a simple framework for generation of synthetic musical compositions for use as training data using the popular Musical Instrument Digital Interface (MIDI) protocol, which may be rendered as audio using commonly available synthesizers. In addition, we apply a variety of audio transformations to the generated audio samples for data augmentation purposes. We apply this music synthesis algorithm to the MIR tasks of polyphony estimation (number of instruments sounding) and instrument recognition (which instruments are playing) in polyphonic tracks where multiple instruments may sound simultaneously in each analysis frame, and evaluate our framework on publicly available annotated music datasets. We empirically demonstrate that pure synthesis of a musical training set without usage of a training set of music yields statistically significant improvements over a random or majority classifier. The main contribution of this thesis is to show that synthetic musical composition generation coupled with data augmentation has the potential to aid content-based MIR in music collections with limited amounts of annotation.