Combining Image Processing Techniques, OCR, and OMR for the Digitization of Musical Books

Digitizing historical music books can be challenging since staves are usually mixed with typewritten text explaining some characteristics of them. In this work, we propose a new methodology to undertake such a digitization task. After scanning the pages of the book, the different blocks of text and staves can be detected and organized into music pieces using image processing techniques. Then, OCR and OMR methods can be applied to text and stave blocks, respectively, and the information conveniently stored using the MusicXML format. In addition, we explain how this methodology was successfully applied in the digitization of a book entitled "The Music in the Santo Domingo's Cathedral". In particular, we provide a new annotated database of musical symbols from the staves included in this book. This database was used to develop two new OMR deep learning models for the detection and classification of music scores. The detection model obtained a F1-score of 90% on symbol detection; and the classification model a note pitch accuracy of 98.4%. The method allows us to conduct text searches, obtain clean PDF files of music pieces, or reproduce the sound represented by the pieces. The database, models, and code of this project are available at https://github.com/joheras/MusicaCatedralStoDomingoIER.