Score following and Music Hack Day Barcelona

At the end of last week, Bram was in Barcelona for Music Hack Day: a two days event focused on music technology hacking (web, software, hardware, art, ...) We launched our MeloTranscript web service API, and also connected up our music following technology to the MuseScore score editing software.

The first Music Hack Day was in London in July 2009, and was organized by Dave Haynes (from SoundCloud) and James Darling. The idea is to have music technology companies provide a set of application programming interfaces (APIs), and then bring together a group of technology enthusiasts/developers to "hack" together new applications based on these APIs within a time span of 24 hours. Typical companies at these events are SoundCloud, The Echo Nest, Last.fm, etc..., but also the BBC has participated in the past.

Bram (at the left) entering the hack room at Music Hack Day Barcelona. Gotta love these logo-stamped T-shirts ;-)
(photo by Thomas Bonte, source: Flickr, license: CC BY 2.0)

In Barcelona, we launched our SampleSumo MeloTranscript web service API.
It's a web service that allows a person or an app to upload a sound recording of some singing, humming, or other monophonic melodic content, have it processed by a pitch analysis and segmentation algorithm, and get back a note transcription result. So basically: sound goes in, note labels come out. More on this in a separate blog post here.

Apart from that, we also demonstrated a music score following system, where the cursor in an electronic score is advanced automatically while someone is playing the piece. The typical use case for this technology is an automatic page turning system: instead of having to manually turn over a score page at the last bar, an electronic score is automatically flipped to the next page when the player reaches the end of that page. We also have a couple of other applications in mind where this technology can be used for synchronizing music to other data.

For this demo, we connected up our real-time music follower to MuseScore, an open source music notation editor package. While music is being played back, our software analyzes the incoming sound, matches that up with the info from the score that is being played by the musician, and sends "current position" messages to the score editor to update its cursor. In this particular setup, we used a video recording of one of the live performances from the Open Goldberg Variations by the pianist Kimiko Ishizaki, but it also works with other instruments.

Score following setup. The laptop at the left is running our music following software and sends synchronization messages to MuseScore, a music notation software package. The iPad at the front is playing back a video of a live piano performance.
(photo by Thomas Bonte, source: Flickr, license: CC BY 2.0)

We also received some interest from the Spanish press: Bram was interviewed by the Spanish national television station Telecinqo and also by BTV, the Barcelona television station, on which the interview and demo was aired live.

The Spanish national television station Telecinqo interviewing Bram about the score following hack.
(photo by Thomas Bonte, source: Flickr, license: CC BY 2.0)

We will be offering this technology component to interested partners in the near future (feel free to contact us if you are interested).

This music following system is one of the outcomes of the R&D project we started last year in collaboration with Ghent University, with support from IWT and funding from PMV.
We have another music performance related technology component in the pipeline, and are currently also working on some promising audio analysis technology for games (and advertisements). But more on that later!