Score Informed Piano
Piano transcription is the process of converting a piece of music originally written for another instrument into a format that can be read and played on a keyboard. Music transcriptions are commonly used to preserve and communicate musical compositions and performances in a standardized form, and have traditionally been performed by trained musicians using music notation software. Transcribing music to a piano requires an understanding of harmonic, rhythmic, and stream structures as well as advanced aural skills to accurately capture the performance. However, it is also necessary to consider what is physically possible on the instrument and what may be influencing a performer’s interpretation.
The ability to transcribe piano music with high accuracy is challenging because of the distinct timbres and playing techniques associated with different instruments. For example, the acoustic properties of strings, hammers, and plectrums can impact the frequency spectrum of notes played on a piano. Additionally, piano performances often involve multiple simultaneous pitches, and a variety of pitch intervals. Therefore, the acoustic model used to train piano transcription systems must be capable of handling these varying characteristics.
Previous approaches to piano transcription have focused on spectral features, but this approach fails to capture the multi-modal characteristics of piano music. In addition, previous onset-only systems have been inconsistent with the timing of note onsets within frames. The occurrence of successive notes with the same pitch can shift the precise onset time by up to 40 ms, which is problematic for piano transcription.
Score Informed Piano Transcription
In order to address these challenges, this work focuses on scoring-informed source separation for piano transcription. To this end, a non-negative matrix factorization (NMF) framework is used to decompose the spectrogram into a set of dictionaries and an activation vector. The spectral patterns S and the piano-roll matrix HB are initialized with score information, and the notes-to-unit matrix E and the unit activity matrix A are updated during the NMF separation phase.
To reduce the complexity of the resulting system, we employ a hierarchical decomposition and optimization algorithm to find the optimal values for the matrices and their derivatives. The result is a piano transcription system with a computational complexity of O(N) and an F-measure score of 93% on the Goldstein test set.
Moreover, the results demonstrate that this method achieves highly accurate transcriptions by learning for each score pitch a set of templates capturing the spectral properties of that pitch, and choosing for each pitch an individualized threshold to differentiate note onsets from spurious activity in an optimized way. Consequently, the performance of our system is competitive with state-of-the-art piano transcription algorithms. These results demonstrate the effectiveness of our scoring-informed piano transcription methodology and its scalability to new musical styles.
