Modularizing MuseScore

TLDR

Benefits of modularizing

• Easier to contribute (consider things in isolation, not hit with everything at once)
• Easier to maintain (well-defined structure + relationship between modules)
• Extensible (swap modules in and out like plugins)
• Portable (remove large dependencies from backend to ease porting to Javascript / compiling to webassembly / mobile apps / Python bindings, etc.)
• Better storage in Git (see https://bitbucket.org/blog/monorepos-in-git)
• Chance to tidy up (fix header files, variable names, indentation, etc.)

The long version

The release of MuseScore 3 is a great achievement on the part of the development team and of the wider community. What was once a small hobby for one individual has grown into a global project of epic proportions. MuseScore has more contributors than ever before, more features than ever before, and, unsurprisingly, more code than ever before!

We are no longer at a point where any one individual can claim to understand every aspect of MuseScore’s codebase. With the increased pace of development it becomes ever harder for the development team to keep on top of things and ensure that everything progresses smoothly and in the right direction. It is vital that we look at improving the structure of the code if MuseScore is to sustain this level of growth in the future.

The solution

What we need to do is separate MuseScore’s code into distinct modules. Each module should operate independently of the others, with communication taking place over an API that remains relatively stable. This allows programmers to treat modules as “black boxes”, passing data in and receiving the output without having to worry about the details of how any module works other than the one they are currently working on. It also allows the code within each module to be improved and iterated upon with minimal impact to other modules. On the rare occasion that it is necessary to make a breaking change to the API, the improved structuring would make the change much easier to implement.

Adopting a modular design would enable developers to focus on their individual area of expertise, without worrying about the implementation details of other modules. Each module could have a designated maintainer responsible for reviewing code contributions to that module ensuring that the API exposed to other modules is preserved. This would also enable additional modules to be created and maintained by the community, with the core team able to decide if and when the community modules should become officially supported.

What about extensions?

The extensions mechanism should help to address the problem by separating out new components that are not needed by everybody. However, extensions exist outside of the codebase, so they can only offer additional resources (instrument sounds, templates, palette elements, etc.), not improved performance or functionality; these are challenges that have to be addressed at a deeper level.

It is possible that the extensions mechanism could be used to deliver dynamic libraries (a.k.a. “binary extensions”) that would indeed add new features, such as support for more import and export formats. However, this can only happen after the modularisation has taken place, otherwise there will be no API for the libraries to interact with.

How it would work

The most obvious candidates for modularisation are:

• Notation dialects (CWMN, Guitar Tablature, Gregorian Chant, Jianpu, Braille, etc.)
• Import and export formats (MSCX, MusicXML, MNX, MIDI, etc.)

The core code would define abstract (virtual) C++ classes for NotationDialect and NotationFormat, which would then be implemented in the modules by the concrete classes CWMN and MSCX, and possibly in GuitarTab and MusicXML classes too. The community would step in to implement the remaining classes in separate unofficial modules, and the core team could choose to:

1. Include the community module with MuseScore out of the box.
2. Provide the module as a binary extension through the extensions mechanism.
3. Run the module as a service on MuseScore.com (e.g. format conversion).
4. Add a build option for people to compile it on their own.

Community role

It may be that the core code defines a basic version of a module that will suit the needs of the majority of users, while the community may provide a more advanced version. An example of this would be SVG and PDF export, which can be provided quite easily by Qt but with limited functionality. Here the community would step in to provide better export facilities (in the case of SVG they already have, though like everything else it could benefit from a more modular design).

In the future, the community might even implement modules for PNG, PDF, SVG and MP3 import, which would call upon machine learning techniques to provide automated OMR and audio transcription facilities. While some efforts have been made, a modular design would make it significantly easier to make progress in this area, as it would allow OMR experts to concentrate on OMR rather than on how to get it working with the rest of MuseScore’s code. Progress in this area would further strengthen MuseScore’s university relations, attracting talented students and research funding.

Benefits

Better integration and reduced code duplication

There are many places where the current monolithic design has led to features being duplicated. An example of this is the MIDI import facility and the realtime input mode, both of which implement conversion of MIDI data into music notation, but do so entirely separately.

A modular design would allow these features to share an underlying pianoroll-like internal data structure, which would then be converted to notation using a shared set of algorithms for:

• quantization (rounding note-on and note-off events to the nearest beat)
• tuplet detection
• rhythmic grouping (adding beaming and ties)
• ornament recognition (detecting staccato, trills, etc.)
• converting velocities to dynamics (p, mp, mf, etc.) and dynamic changes (cresc., dim. etc)
• converting MIDI tempos to tempo changes (rit., rall. etc).

Improvements to these algorithms would benefit MIDI import, realtime MIDI input, and a potential transcription-from-audio feature as outlined in the previous section.

Improved portability

Basically we should completely remove Qt from the backend. This would enable:

• Easier porting to other platforms (e.g. mobile apps, compile to Javascript / WebAssembley)
• Separating the command line interface from the GUI
  • no need for a GUI for rendering audio and PDF / SVG
  • Don’t need a synthesizer to generate PDFs
  • Don’t need layout to generate audio
  • Simple operations on MSCX files don’t require any of the above

MuseScore as a platform

MuseScore contains many features designed to suit a number of different use cases, but it not possible for a single program to keep everybody happy. The program is large and complicated enough already, but a modular design would allow new features to be developed outside of the existing codebase. This, combined with a binary extensions mechanism, would enable users to pick and choose the features they need.

Wordpress has built an entire ecosystem of open source plugins and themes, and MuseScore could do the same for music notation. It is not hard to imagine a MuseScore “extensions store”, which would operate a bit like an app store, allowing the user to choose extensions that add new features for editing scores, or specialised playback facilities such as a karaoke mode. The desktop software itself would basically just be a client shell with hooks for extensions, a bit like a fresh Wordpress installation. Of course, just like with Wordpress, MuseScore would come with various extensions installed by default, so it would remain a fully functional program out of the box.

MuseScore as a software library

Developing a program for multiple operating systems is a huge challenge that wouldn’t be possible without a framework like Qt. Developing music software that can support multiple notation formats is equally challenging. As the leading open source notation program, MuseScore is already the defacto standard for converting MusicXML to PDF and other formats, so why not expose this ability to other open source programs via an API? A modular design would also make it possible to create bindings to other languages like Python, in addition to C++.

Transition period

Development is a learning process, and it is only by trying things out and making mistakes that you learn the right way to do things. However, the lesson often comes when it already too late to change what you are doing, and you are forever stuck doing things the wrong way. We have seen this with Finale and Sibelius, the two dinosaurs of music notation, and we are already starting to see it in MuseScore (a prime example of this is local time signatures, which is forced to use a very hacky implementation due to the way things are implemented at a lower level in the code).

There comes a time in the development of any software application when you basically need to start all over again and reimplement everything from the ground up, drawing upon the lessons learned the first time around. We have seen from Dorico the transformation that can be achieved when you hire a bunch of developers and tell them to “make the program you wish you had made 10 years ago”. MuseScore clearly beats Dorico in terms of price, but if we are to compete on a feature level then it is essential that we re-examine the core structure of our own codebase.

Fortunately, we don’t literally have to start from scratch. One of the benefits of a modular design is that we can create the modules one-at-a-time, and gradually integrate them into the codebase as they mature. Many of MuseScore’s code files are already partially modular, so in these cases all that is needed is some virtual classes (i.e. the API) and a better directory structure.

Possible modules:

• Muse: Abstract music module implemented by dialects (pianoroll / MIDI with fractions)
• Score: a dialect (CWMN, Jianpu, etc) + simple operations like inserting/deleting notes, joining scores etc. (computationally cheap)
• Layout: operations involving visual rendering of musical symbols (computationally expensive)
• Synth: operations involving audio rendering for playback (computationally expensive)
• Model: data construct for GUIs (palettes, templates, etc.)
• Viewer: GUI that uses the Model to provide playback facilities
• Editor: extend Viewer to provide editing features