AI-Assisted Annotation and Analysis of Music Performance

In the internet era, vast troves of sound recordings and musical scores have become available to virtually anyone, but drawing connections between these two mediums to develop insights about performance style and artistic expression has remained a difficult and time-consuming task. Currently, music scholars rely on general­-purpose sound editing tools for annotating musical elements in recordings (such as downbeats, pitch, or note onsets); however, these annotations often must be entered manually, and these tools do not incorporate one vital element needed for analyzing recordings: musical score information. This approach can be labor intensive and inefficient, and limits the scope of the data that can be analyzed. To help address this need, we will develop a prototype open­-source software tool for annotating and analyzing recorded music performances, bringing innovations from the interdisciplinary Music Information Retrieval (MIR) community to scholars and musicians in the humanities.

Provided with a (digital) musical score and a performance of that score, the software will automatically generate meaningful AI­-driven annotations about note onsets and tempo, fully integrated with the score. With an easy­-to-­use graphical user interface, the software will display and visualize the score, the audio, and the selected annotations in intuitive ways designed to streamline performance analysis. Besides developing the prototype, we will conduct use-case studies, gathering evidence to illuminate ways such a tool can be used to assist musicology scholarship, and to inform future development of the software.