MusicXML Import Problems

• Mar 10, 2019 - 04:38

It's clear that Musescore does not fully implement MusicXML page and system formatting as described in the latest MusicXML specification (3.1). System width, measures-per-line and optimized (collapsed) systems are ignored. Also, certain MusicXML attributes (chiefly "Instrument Change") appear randomly as TEXT instead of being applied to the underlying MIDI events. I am posting 2 examples (6 screenshots) here:
1: After the Love Has Gone
ATLHG_SS.jpg: Original TIFF and SmartScore interpretation
ATLHG_FIN.jpg: Finale interpretation of SmartScore's XML output
ATLHG_MS.jpg: Musescore interpretation of SmartScore's XML output
Musescore Problems Observed:
- Optimized system (1st system) should not display vocal line.
- Number of measures per system unsupported
- Number of systems per page unsupported

2: Blue Eyes
BE_SS.jpg: Original TIFF and SmartScore interpretation
BE_FIN.jpg: Finale interpretation of SmartScore's XML output
BE_MS.jpg: Musescore interpretation of SmartScore's XML output
Musescore Problems Observed:
- MIDI attribute ("Instrument Change") appears as text throughout
- Optimized system (1st system) should not display vocal line.
- Number of measures per system unsupported
- Number of systems per page unsupported

To provide some background, the following is from an exchange I had with "Musescore Education" on Facebook:

Me: Finale XML files imported into MusicScore have serious formatting issues. That's also true with SmartScore XML files. The XML specification was developed and is maintained by the folks at Finale / MakeMusic.

Musescore: MusicXML doesn't really excel at conveying formatting, so many applications choose to ignore it rather than try in vain to mimic the formatting of a different program. Think of MusicXML as being a way to transfer the musical content, but then rely on the importing program to do its own formatting (and allow you customization over it). Looked at this way, MuseScore has some of the best MusicXML impot in the business.

Me: I have to disagree. MusicXML describes in detail how the music should be formatted on the page. Why NOT conform to the how the page is formatted within the exporting program? It makes less sense to force the music to fit into the arbitrary default layout of the importing program. I have examples if you provide me with your e-mail address.

Musescore: It describes this in theory, but the reality is often quite different. Much depends on how any given program decided to space notes, positions that make sense in one program might not in another. One program can fit six measures of a given piece on a system, another can only fit five, and that throws everything off, etc. Anyhow, if you're expecting perfect reproduction of the formatting, you're going to be disappointed, I don't know of any programs that guarantee this, and we definitely don't make that a goal. Instead we focus on import of the content itself.
Anyhow, if you have specific examples where you think there is room for improvement despite this, feel free to start a discussion on the Support forum on musescore.org

Me: It describes "in theory"? No Messieur. It describes in DETAIL. Musescore simply chooses to ignore those features. I must ask you if you are familiar with how the Musescore XML import module is coded. Are you close to the development team? Can you kindly bring any of this to their attention?

Comments invited.
Kit

Attachment Size
ATLHG_FIN.jpg 1.96 MB
ATLHG_MS.jpg 476.75 KB
ATLHG_SS.jpg 1.22 MB
BE_FIN.jpg 1.93 MB
BE_MS.jpg 624.97 KB
BE_SS.jpg 766.37 KB

Comments

For the record, that's me - I'm the Director of Education for MuseScore.

The formatting information in a MusicXML file might be detailed indeed, but that doesn't mean it is actually technically feasible for all programs to reproduce all of it in all cases, nor does it mean it is worth the enormous effort it might take to rewrite layout algorithms to accommodate this. For a small open source project like ours, it's a question of priorities. And yes, I am familiar with the code, being one of the developers myself.

Right now, we simply don't support any way to force more measures per system than what would fit naturally given your particular settings for staff size, measure spacing, minimum note distance, etc. So we simply have no mechanism by which we could support a MusicXML tag/attribute specifying a particular number of measures on a particular system. This could indeed be an interesting new feature to add, not just for MusicXML import, but for general use as well. But for now, no way to implement this on MusicXML import without significant changes to the layout algorithms. We can honor system breaks because we have tha feature, but we cannot honor a "fit X number of measures on this system" because we just don't have that feature, MusicXML or no. And the same is true of number of systems per page - we simply have no ability to set this in the program, only to control various parameters like staff size, staff distance, and minimum system distance, that ultimately give you that control. So if a MusicXML file specifies that, we could potentially honor it, but if all it does is say "give me three systems on this page", we simply don't have that facility.

Similarly, we don't have any facility to identify particular systems as optimized or not. We have a global "Hide empty staves" setting that applies to the score as a whole. So again, no way to honor a per-system setting in MsuicXML without significant changes to the layout algorithms.

Instrument changes we do support, via text instructions, which of course the human musician reading the printed score would need as well. You are welcome to make them invisible. I have no idea why your MsuicXML file has so many of these instructions encoded, seems likely to be a bug in the program that exported them since there is nothing I can see in the original scanned image to indicate any such changes. I'd need to see the MusicXML file and understand more about the context to understand.

Anyhow, this is the sort of thing I mean when I say that "in theory" the MusicXML can contain certain information, but in reality, there may or may not be a way to honor it. If the information encoded happens to be the sort of thing the receiving program is equipped to deal with based on its own feature set, great, but not all programs have the same feature sets, so information encoded by one application just might not be relevant for another.

Of course, Finale can generally import the information its own products export, and since they controlled much of the original spec, they also had a say in what others could export, so they'll generally do better than average there as well. Even so, there are likely some things others export they don't handle because it's outside the feature set of Finale itself, and this will probably become increasingly true as their control over the MusicXML spec has become less absolute.

So again, it is indeed true we, along with many others, do not attempt to interpret much manual formatting info on import, focusing more on content.

In reply to by Marc Sabatella

Totally agree with Marc here. Without access to the MusicXML files it is impossible to judge if MuseScore's import should be improved. If you could attach or mail these I'd be willing to investigate. And for your information, I am the developer of almost all of MuseScore's MusicXML import / export code.

In reply to by Leon Vinken

Hi Leon
Kit Newell from Musitek here. I just noticed your follow-up post.
As I mentioned to Marc, we get a lot of heat from Musescore users that our XML output isn't any good. We're forced to defend our products and then direct them to Noteflight for their online / Java-based notation needs.
Our Music-to-XML desktop gadget converts PDF and TIF music image files to XML. It's become quite popular due to its accuracy and simplicity. We'd love to see better M2XML and SmartScore integration with Musescore. I'd be happy to provide you with a copy of Music-to-XML and SmartScore as well if you're interested.
Glad to assist you in any way I can. Here are the sample XML files output from SmartScore and Noteflight's interpretation (same files sent to Marc).

Cheers
Chris (Kit) Newell
Musitek

Attachment Size
Blue-Eyes.xml 284.31 KB
The Way It Is.xml 447.46 KB
BE_NF.jpg 184.57 KB
ATLHG_NF.jpg 175.73 KB

In reply to by Marc Sabatella

Here are the two XML files exported to (and properly interpreted by) Finale and Noteflight.
NOTE: You stated: "Of course, Finale can generally import the information its own products export, and since they controlled much of the original spec..."
In fact, the program that exported these XML files was SmartScore by Musitek; a totally different company. The export engine was built by Musitek purely from the MusicXML spec with no assistance from MakeMusic.

To your point about formatting (measures per system, systems per page, etc.): You say "that doesn't mean it is actually technically feasible for all programs to reproduce all of it in all cases."...
I am including screenshots of Noteflight's interpretation of both XML files. As you can see, systems are optimized and measures per system are retained. You will have to admit their results are superior to yours and, obviously, implementation is feasible.

To clarify... I do technical support for Musitek. We get quite a few calls from Musescore users who complain that our Music-to-XML gadget is lousy. Many demand refunds. We are forced to defend the product and direct them to Noteflight. Improving your XML import function would be a boon for both our companies. I can arrange to have you set up to download a copy of Music-to-XML if you wish.
Thanks
Kit Newell
Musitek

Attachment Size
Blue-Eyes.xml 284.31 KB
After the Love.xml 535.76 KB
ATLHG_NF.jpg 175.73 KB
BE_NF.jpg 184.57 KB

In reply to by Kittifer

I didn't say it's not feasible for anyone to implement these feature, just that it isn't necessarily feasible for everyone to. If the feature happens to exist in the receiving program, it's easy. If not, it's hard. I suspect it would not take me long to find things we import better than others, but anyhow, my overall point remains that with the limited resources the volunteers who develop MuseScore in their free time have available, we focus more on content then formatting. Not saying that situation cannot ever change, but for now, that's how it is.

one reason for this focus is we are trying to be future-focused - a future in which more and more people use electronic devices for reading scores. We don't want them locked into the original formatting that only made sense at that exact page size. Having responsive layout that adjusts well to different display sizes and magnifications is very important to us, so more development effort goes there.

Meanwhile, you can also suggest they simply deleting the breaks and turn on Format / Style / Hide empty staves". Results won't look identical to the original but will be perfectly valid representations fo the same score. Feel free to direct users who complain about the quality of the import to these forums and we're happy to explain the same to them.

Also, from a practical perspective: my guess is you could improve the results from MuseScore actually setting whatever parameters for staff size and default note spacing MuiscXML provides. But Leon is definitely the expert, so I'm glad to see the conversation broadening. I definitely don't want to give the impression I speak for MuseScore as a whole.

In reply to by Marc Sabatella

An update from my side. I did check your MusicXML files and found no issue with them. They are completely valid. As for why the import into MuseScore produces sub-optimal results (skipping the issues already explained by Marc):

The most visually obvious issue is probably the large number of "Instrument change" texts generated. These are actually real instrument changes (they DO affect MIDI playback, if they don't this is a regression), they are not simply text. The instrument changes are present because:
- MuseScore generates these whenever a note is played on a different instrument than the previous note used
- Your MusicXML files use a different instrument for every voice in the same part (rather unusual in MusicXML), which is not expected nor handled correctly by MuseScore. I consider this a MuseScore bug, which unfortunately is not entirely trivial to solve. The simple workaround is to select all of them and delete them at once.

The second obvious issue is the changed number of measures per system and number of systems per page. This happens because there is no obvious way for a MuseScore importer to force the MuseScore layout algorithm to use a specific position for all imported elements. The layout algorithm itself determines element positions based on generic style settings and element sizes. Thus most layout information present in MusicXML has to be ignored. Exceptions are generic things like page size and margins. MuseScore may (as a user preference) import system and page breaks. Typically this works reasonably well, but sometimes this produces unexpected results. When the Musescore layout algorithm calculates much wider measures than the exporting application did, systems containing only a single measure are produced. In that case, tweaking the MuseScore layout parameters may help.

In reply to by Leon Vinken

Knowing what I do about the way the layout algorithm is implemented, I would say it could actually be feasible without a ton of effort to implement a "fixed" width for measures, at least if you promise not to set it less than the calculated minimum for the actual content. This could be useful also to re-implement an option we had long ago to force a consistent measure width within each system. It never worked well then which is why it was removed, but I don't think it would be hard to add it back. And then MusicXML import could use the same underlying facility.

In reply to by Leon Vinken

Thank you for the affirmation, Leon. We have been comfortable with our implementation of MusicXML for years having worked closely with Michael Good (MakeMusic) who developed the format and have tested it on many platforms, including Noteflight which imports and interprets our XML files quite well.
Let's get a fix in the works... for Musescore's sake and for ours !

In reply to by Leon Vinken

Its sometime since the original post, but I do not agree that the issue with the "instrument change" is a MuseScore problem. At least for braced staves (Grand Staff as used for piano) SmartScore should only use one instrument. It would be correct to use multiple instruments for bracketed staves (like for Violin and Viola).

I would expect that at least the 'Piano Edition' which I'm using, does it correctly. Don't they play the piano at Musitek?

I found no way to tell SmartScore that the Grand Staff should only use one instrument.

Do you still have an unanswered question? Please log in first to post your question.