Wolfram Language

Text Recognition to Read Music Sheets

In the world of music, tab notes (or tablature) are a way to notate the instrument fingering.

This example shows how to use text recognition to read the notations from tab notes and generate a simulation of the music being played.

Take one piece of a tablature from a Led Zeppelin song.

To locate the text, start by finding bounding boxes corresponding to each row of notes. The classical image processing approach is enough to perform the task.

show complete Wolfram Language input

You can visually confirm that the localization is pretty accurate.

Now, having the text locations pre-computed, you can start the recognition.

show complete Wolfram Language input

You will need to clean a possible noise from the recognition; for example some "0" characters may be interpreted as "o", depending on the font used.

Now convert the strings to lists of notes and remove the last character of each line, which is a stop character.

show complete Wolfram Language input

In order to have a piece of music, you now need to convert the lists to chord lists and apply a proper tune.

show complete Wolfram Language input

Now you can put all the steps together to recognize a Led Zeppelin song.

show complete Wolfram Language input

Related Examples

de es fr ja ko pt-br zh