The Con Espressione! Exhibit is an interactive system designed for popular science exhibitions. It demonstrates and enables joint human-computer control of expressive performance: the visitor controls overall tempo and loudness of classical piano pieces (such as Beethoven’s “Moonlight” sonata Op.27 No.2) via hand movements (tracked by a LeapMotion sensor). In the background, our “Basis Mixer” expressive performance model adds subtle modifications to the performance, such as articulation and micro-timing (e.g., slight temporal differences in the note onsets when playing chords). The contribution of the Basis Mixer can be controlled and experimented with via a slider.
The exhibit was first shown at the la.la.lab Science Exhibition (“The Mathematics of Music”) in Heidelberg, Germany (May - Dec. 2019). The source code is openly available via a GitHub repository.
The Con Espressione! Exhibit: Exploring Human-Machine Collaboration in Expressive PerformanceIn Late-Breaking and Demo Session of the International Conference on Music Information Retrieval (ISMIR), 2019