Can mode be specified in a key signature?

• Apr 13, 2018 - 19:45

I'd like to be able to specify the mode with a key signature—that is, when I put in a key signature of (say) 2 sharps, I'd like to be able to specify whether it's D major or B minor (or E Dorian, etc...). This is supported in Finale, ABC, LilyPond, and MusicXML, but I haven't yet found a way to do it in MuseScore. Is there one?

I do note that once MuseScore has imported a MusicXML file, it seems to keep the mode information intact—that is, I have some MusicXML files with a key of G minor that were imported, and MuseScore seems to know that they're not in Bb major. But I can't figure out how to do this for newly created MuseScore files. Help?

Comments

• Apr 13, 2018 - 21:17

Hmm, looking at the XML content of the .mscx file, I see that it has:

        <KeySig>
          <accidental>-2</accidental>
          <mode>minor</mode>
          </KeySig>

But I can't figure out how to set the <mode>minor</mode> bit from the UI.

Reply

• Apr 14, 2018 - 00:28

MuseScore makes no distinction between modes, which is why there is no way to set it - such a control would have no effect. I suppose it could in theory be taken into consideration when deciding whether to spell a note entered via MIDI as A# (very sensible in in B minor) or Bb (arguably more sensible in D major), but that's about the only thing I can think of. What is the actual problem you are trying to solve here? Maybe you are planning to export to MusicXML and you just want the info there?

Reply

• Apr 16, 2018 - 01:10

> MuseScore makes no distinction between modes

Then why does it store the mode in the MuseScore XML file? Surely any data stored in the MuseScore document should be (somehow) accessible from the GUI, no?

> such a control would have no effect.

That's probably true as far as the ultimate layout of the notation, but considering only that is IMHO shortsighted, for several reasons. Read on. :)

  • First of all, you yourself point out that it actually would be very useful for controlling enharmonic spellings. Finale uses the corresponding setting for this exact purpose, for what it's worth.

  • Second, MuseScore doesn't only generate notation graphics; it also generates other formats including MIDI and MusicXML, and a mode control would have an effect on those outputs from MuseScore.

  • Also, the key (as opposed to the key signature) is important metadata about a piece of music or section, and every other music format that I know of provides a way to store it. MuseScore should too. (I'll note that there is already precedent within MuseScore for considering the semantics of keys separately from their signatures: MuseScore does have separate options for C major and neutral key [for atonal music].)

> What is the actual problem you are trying to solve here? Maybe you are planning to export to MusicXML and you just want the info there?

Well, it's a little more complicated than that. Yes, the proximate issue is that of MusicXML export: if I cannot specify that a piece is in G minor rather than Bb major, then the resulting MusicXML file will be semantically incorrect, and that's just not acceptable to me. (For the minor mode, I can manually edit the MSCX file before the MusicXML export takes place, but that doesn't work for Dorian—I've tried.)

Let me back up and explain my current project, and how MuseScore and key issues fit into it.

I'm producing a Web edition of a book of klezmer tunes, programmatically converted into various formats (PDF, MusicXML, MIDI, ABC, etc.) for ease of use. I am automatically generating tables of contents from the title and key metadata present in the tune source files.

When I started this project, I was using ABC as the source format. It was trivial to parse the T: and K: headers in the ABC files to get the title and key metadata I needed.

ABC proved too limited as a source format, and so I started using MusicXML as source format. Again, it was quite easy to parse the MusicXML to get the metadata.

But then MusicXML in turn proved too limited as a source format, so now I'm using MSCX as source format, and converting to MusicXML and other formats. Unfortunately, while MSCX can satisfactorily store all the other information I need to store, it cannot store key (as opposed to key signature). This means that I no longer have access to the correct key metadata to build the table of contents.

Worse, the problem propagates. Since MSCX can not store proper key information, any MusicXML file generated from the MSCX source files is incorrect with respect to key, even though the MusicXML format itself could store the right key. Since my tooling then generates other formats such as ABC from the MusicXML, rather than directly from the MSCX file, those files are in turn incorrect in the same way: the key metadata is discarded because MuseScore simply can't store it—even though all those other formats could store the correct metadata, they do not get the opportunity to do so.

From my point of view, then, it is pretty much essential that MuseScore store key information, both for reasons of interoperability with other formats such as MusicXML and for proper storage of metadata so that the computer can properly reason about the semantics of the music contained in the file. (It would also make things like key transposition work better—see the corresponding feature in Finale with respect to transposing across modes.)

Frankly, if MuseScore doesn't provide a way to store this data, I don't know how I'm going to complete my current project. Open-source MusicXML authoring tools are hard to come by (and MusicXML is not human-writable!), and MuseScore has all the other features I need for this project, but it's absolutely essential for the reasons given above that we figure out how to get keys right. If my C++ were better, I might try to submit a pull request, but my programming expertise is mostly elsewhere...

Anyway, I know that was longish, but I hope it at least puts more context on why I think MuseScore needs this feature. I'd be interested to continue this discussion.

Reply

• Apr 16, 2018 - 01:40

MuseScore allows you to add metadata to a score as you have probably seen. You can add a new item in the score properties perhaps named "Mode" and enter the mode yourself in this spot. It will be consistent in its location in the file for you to find it for your project.

Reply

• Apr 16, 2018 - 04:38

If you're concerned with every mode of every key in the piece it would be interesting to find a program that actually keeps track of that. Even if the key doesn't change it's not at all uncommon for the mode of the key to change, especially from minor to major without a key change.

Reply

• Apr 16, 2018 - 07:33

On MusicXML import (and apparently export too) MuseScore accepts and (sort of) handles the modes major, minor and none: mscore/importmxmlpass2.cpp. lines 3476-3491.
It could/should probably handle all modes MusicXML allows for: mscore/schema/musicxml.xsd. line 704, so major, minor, dorian, phrygian, lydian, mixolydian, aeolian, ionian, locrian, and none, at least on import and export of MusicXML. Whether and how to handle them inside MuseScore is yet another issue, could be some right-click context menu or probably better the key signature inspector. And then it needs to be handled by the note entry code in regards to pitch spelling, I guess

Feel free to enter these as separate feature requests into the issue tracker.

Reply

• Apr 16, 2018 - 17:38

I do get that a feature to allow specifying mode could be useful for certain purposes. I am merely trying to explain how and why things are currently. You ask why we write mode info to the XML file, but I'm confused. We don't, normally. Only if the info was already present due to import from a MusicXML file that specified it do we do this. Preserving it is harmless, so why not? But we don't write mode info to MusicXML unless there was already mode info present, because as mentioned, we simply don't have any means by which a user could specify a mode, nor are there any behaviors within the program that would currently be affected by mode. It's not a bad idea for the future though, for exactly the reasons you suggest, so feel free to open a feature request in the issue tracker for this.

BTW, the difference between C major and "open" key signature is vital because it does affect behavior within the program - it controls what happens during transposition.

However, I will strongly caution you against using MSCX yourself. This format is constantly changing, and deliberately not documented so that use of it outside of MsueScore is not supported. You will be much better off going back to MusicXML. For your project, I'd suggest using MuseScore for the inital entry of the data, then generating the MusicXML, hand-editing it (for now) to add the missing metadata, and then treating that MusicXML as your primary source henceforth.

Reply

• Apr 16, 2018 - 17:56

I'll respond to the rest of your post later, but I did want to get one point out of the way.

> However, I will strongly caution you against using MSCX yourself.

I'm not doing anything directly with MSCX except converting it into MusicXML (and PDF); everything else operates on the MusicXML. But in order to get a proper conversion to MusicXML, I may need to hand-edit the key info in the MSCX file, until such time as MuseScore provides a way for me to do that in the UI.

> then treating that MusicXML as your primary source henceforth.

I explained in my initial post that I tried to do that, but couldn't, because I am storing data that MusicXML cannot completely represent. Because of that, MSCX is my primary format for this project and will probably have to remain so.

Reply

• Apr 16, 2018 - 18:03

I guess I'm not really understanding how to reconcile these statements. Maybe we mean something different by "primary format". Or I am confused by why you refer to MSCX instead of MSCZ. I was assuming you meant, you have tools designed to parse and work with the MSCX format directly. This is what I am saying is a bad idea. If all you do with MSCX is convert to MusicXML, and everything else operates there, then to me that isn't saying MSCX is "primary", just that it happens to be what you are using for initial input. And you could just as easily be talking about MSCZ at that point. So I guess I am still a bit confused.

Reply

• Apr 16, 2018 - 18:37

My Git repository for this project contains an MSCX file for each tune. (I could just as easily have used MSCZ, but text formats are a little friendlier to version control than binary.) All other formats that I generate (MSCZ, PDF, MusicXML, ABC, eventually MIDI) are created automatically by my build tools, ultimately from the MSCX file. (MusicXML is used as an intermediate representation for all formats except MSCZ and PDF, but that doesn't make it primary.)

So the MSCX file is the "single point of truth". It is the only format stored in the repository, and everything else is generated programmatically from it (usually with an intermediate step of MusicXML). That's why I call it the primary format, not because I'm parsing it directly.

Reply

• Apr 16, 2018 - 18:23

the difference between C major and "open" key signature is vital because it does affect behavior within the program - it controls what happens during transposition.

And there we do write <mode>none</mode>. This is as far as I can see the only place were we do use is actively (i.e. not just pass thru what we imported)

Reply

• Apr 17, 2018 - 19:57

> You ask why we write mode info to the XML file, but I'm confused. We don't, normally. Only if the info was already present due to import from a MusicXML file that specified it do we do this.

Right. As explained elsewhere, I converted my source files from MusicXML to MuseScore format, so the mode information came along. It was only when I started entering music that hadn't first been in a MusicXML file that I came across this issue.

> Preserving it is harmless, so why not?

Except that MuseScore doesn't always preserve it. It preserves major and minor, but deletes any other modes that were in the MusicXML (e.g. dorian), which implies some conscious choice on the part of MuseScore's developers, not merely a pass-through. (Do you know why this decision was made? It seems really odd.)

> we simply don't have any means by which a user could specify a mode

Right, I know that now.

> feel free to open a feature request in the issue tracker for this.

Yup, will do.

> However, I will strongly caution you against using MSCX yourself. This format is constantly changing, and deliberately not documented so that use of it outside of MsueScore is not supported.

As I said in another post, I'm not actually doing much of anything with the MSCX files directly, so no fear. But in principle, I think discouraging people from working with the MSCX files is pretty problematic. After all, the format is an XML application—so it's pretty easily parseable—and there are ways of making such things flexible without making them fragile. MusicXML can't capture all the details that MSCX can, so although I'm not doing it for this project, I can easily imagine situations in which I would absolutely want to manipulate the MSCX file. MuseScore is OSS, and should behave as such with respect to its file format as much as with its source code.

But that's another rant and I don't want to derail this thread. :)

Reply

• Apr 17, 2018 - 21:19

To be clear: the fact that we don't preserve eg "dorian" doesn't change the fact that preserving "major" and "minor" is harmless, correct? The point being, there is no bug, just a missing feature. No idea why the other modes were omitted, probably just programming laziness (someone would have had to set up and initialize a larger data structure).

As for MSCX, I don't think you'll find much buy-in for the idea of MsueScore documenting and supporting the use of the format for third party tools. Publishing the specs for a format is a Big Deal, taking a ton of effort that could be spent adding features or fixing bugs, and having the format pubished publicly makes it much harder to make changes and enhancements to it later, thus further limiting our ability to add new features or fix bugs. MuseScore is indeed open source, and you're welcome to study the source to learn how the MSCX format works, but that is no guarantee there won't be changes the next release - we need the freedom to do that in order for MuseScore to continue to improve.

Reply

• Apr 18, 2018 - 08:03

> To be clear: the fact that we don't preserve eg "dorian" doesn't change the fact that preserving "major" and "minor" is harmless, correct?

Correct, but it seems inconsistent to preserve some modes but not others.

> The point being, there is no bug, just a missing feature.

I think you're correct here, although I could easily see calling this a bug because it's so weirdly inconsistent.

> No idea why the other modes were omitted, probably just programming laziness (someone would have had to set up and initialize a larger data structure).

This makes no sense to me. It's actually easier to just pass through any content than it is to pick two modes, so I'm not sure how laziness could have been the issue.

> I don't think you'll find much buy-in for the idea of MsueScore documenting and supporting the use of the format for third party tools.

That's a huge pity, and greatly limits the utility of MuseScore for advanced tasks. (I could theoretically envision a nice rich ecosystem of tools that manipulate MSCX directly to do things that aren't easy from the GUI...)

I guess a couple of years of LilyPond has spoiled me for GUI editing of music, as much as I think MuseScore has mostly done a good job with its GUI. :) (FWIW, the main reason I'm not using LilyPond for this project is that it has poor MusicXML export.)

> Publishing the specs for a format is a Big Deal, taking a ton of effort that could be spent adding features or fixing bugs

I'm not sure you're right about this: the expertise required is different from that required for MuseScore development, so different people could do it—it's a documentation task and could be done by people who would not otherwise contribute code to MuseScore. Thus, it doesn't take time away; rather, it (potentially) gets more of the community involved, which (other things being equal) is probably a good thing.

I should also note that, at least for myself, I don't think any file format documentation beyond perhaps the most minimal (and a test suite) is necessary. What I'd like to see, though, is for the community to stop quite so strongly discouraging playing with MSCX/MSCZ files. :)

> having the format pubished publicly makes it much harder to make changes and enhancements to it later, thus further limiting our ability to add new features or fix bugs.

Not necessarily. It's pretty easy to have a public spec but still allow breaking changes: public format specs change all the time. The file format already contains version metadata that would help this be possible.

I'm not talking about ISO-level specification, just a basic idea of "these are the basic elements, here's an example or two, don't expect anything you do to work after a non-patch version upgrade".

> but that is no guarantee there won't be changes the next release - we need the freedom to do that in order for MuseScore to continue to improve.

Yes, I know that. I have no problem with there being breaking changes in a properly versioned format, so to me, it seems silly to say "don't manipulate this because it might break". Of course it might break, but my (hypothetical) tools can worry about that when it does break, or do a semver-style check on museScore/@version and throw an error if I don't recognize the number.

And now I should probably stop going on about this topic in this thread. :)

Reply

• Apr 18, 2018 - 17:18

FWIW, this has nothing to do with language used. It's the simple fact that while on the surface it might look like MsueScore simply reads in the MusicXML file and then spits it back out so all we have to do is pass information through, the reality is extremely different. Everything read in - whether from MusicXML or MSCX - needs to converted to and stored in the internal data structures used within MuseScore, and then converted to MusicXML or MSCX or whatever when writing. Obviously, MSCX is designed to reflect those internal data structures well, MusicXML requires much more conversion on either end. So regardless of whether MuseScore was written in C++, Python, Java, or BASIC (!), there would still be the need to do all this work.

Reply

• Apr 18, 2018 - 17:29

> FWIW, this has nothing to do with language used.

Actually it does, though that's not the only issue. See below for more on this.

> Everything read in - whether from MusicXML or MSCX - needs to converted to and stored in the internal data structures used within MuseScore

Well, of course it does. I was just surprised that it did anything at all with the mode rather than simply storing it as an opaque string and writing it back out untouched. (I question the wisdom of doing it that way so long as MuseScore takes no action based on the mode, but that's another debate.)

> So regardless of whether MuseScore was written in C++, Python, Java, or BASIC (!), there would still be the need to do all this work.

Yes, but many other languages would remove a lot of the repetition and make a general passthrough easier to implement—something like (Rubyish pseudocode) mode = get_constant(MusicXML::#{mode_string.to_uppercase}) instead of C++'s case "ionian": return MusicXML::IONIAN; break; (which I can't think of an easy way to shorten). That's the point I was making. It's a minor, probably irrelevant, issue in this context, but I did want to set the record straight.

Reply

• Apr 18, 2018 - 17:45

If we were fine with just saving the string, then obviously the code could be simplified further, whether C++ or some other language. But there is value in having things be stored in a more structured way - in particular, while we might not currently do anything else with this info, we might in the future, so having it represented properly in the first place is a win.

No doubt some other language might save a few lines of code here or there, but either way, to do it right requires handling of each case. Since there was no specific value within MuseScore to this, it's understandable it hadn't been done yet. Bu hopefully now that will be rectified thanks to Jojo :-)

Reply

• Apr 18, 2018 - 17:53

> while we might not currently do anything else with this info, we might in the future, so having it represented properly in the first place is a win.

I'm actually going to disagree with that somewhat. Since MuseScore doesn't currently do anything with the mode, there's no way of knowing at this point what the "proper" representation of it might be. In this case, it turns out that an improper "proper" representation was chosen, which made the system work worse than a straight pass-through would have. That's not a win at all from where I sit; it's just some premature overengineering that actually kind of caused problems. Better to figure out the "proper" representation of something when it's clearer what needs to be done with it, no?

I could understand this decision if the data format were frozen, but we've already been talking about how that's not the case.

Again, we're going off into the weeds here...

Reply

• Apr 18, 2018 - 17:27

As you say, MuseScore is open source, so there is absolutely nothing stopping you or anyone else sufficiently interested from spending your own time studying the source and documenting the details of the MSCX format, then hopefully agreeing to update the documentation every time a change is made. But while you say you personally don't mind if the format changes and you have to rewrite portions of your apps each time this happens, not everyone is so understanding, so hopefully you'll also be available to help others with their apps :-). This is a bit tongue in cheek, of course, but the point is, we don't say things like this lightly - years of experience in software development have taught these lessons.

Also, it's funny you mention LilyPond and its poor MusicXML support in this context. The facts that the LilyPond spec is published but keeps changing are directly related to its MusicXML story.

Anyhow, if you look at the MSCX file, you'll see its pretty self-evidence what things do, so it really isn't clear what value a cursory documentation could achieve over simply perusing the file. So I'd say you can already do whatever you might need to do.

Bottom line, if you personally promise you complain in the slightest when the format changes over time, and are cool with contributing to the effort to document the format publicly, I promise I won't advise you personally against it. It's still solid advice for the average person, though :-)

Reply

• Apr 18, 2018 - 17:47

> years of experience in software development have taught these lessons.

I agree with the problem, but it seems to me that the lesson to learn is to version the format (which I see you've done!) and make clear that it's subject to change, not simply to say "it's scary in there, don't touch it".

> The facts that the LilyPond spec is published but keeps changing are directly related to its MusicXML story.

Really? I haven't closely followed the development effort, but I'd be surprised if the file format had anything all to do with it.

The reason I'd be surprised is that, due to Lily's incorporation of Scheme, .ly files are Turing-complete—that is, whereas MuseScore source is a static representation of the music, LilyPond source is (in principle) not itself a representation of the music, but rather a program that generates a representation of the music (like a TeX file). This means that to parse an arbitrary LilyPond file, you need the full Lily/Scheme interpreter, so the .ly input file format is probably the wrong starting point for any sort of conversion. The right starting point for conversion in LilyPond would seem to be the output from the interpreter—that is to say, the live data structure in memory, or a text dump of it (which I think LilyPond can provide). (This is in fact why I haven't tried to improve Lily's XML export: I could write a .ly file parser easily, but working with internals is another matter.)

All of this is a long way of saying that I doubt that the .ly file spec or lack thereof would have anything to do with MusicXML export (but that's a guess), and that .ly files are not really comparable to .msc* files anyway.

> Anyhow, if you look at the MSCX file, you'll see its pretty self-evidence what things do

It looked like it, but with all the warnings, I was afraid to believe that that was the case. Thanks for the reassurance.

Reply

• Apr 18, 2018 - 18:08

The fact that LilyPond has changed details of their format over time and thus adversely affected all sorts of third party tools that work with the format is well-documented. As a result, the developers of those third-party tools have struggled to keep them as full-featured and up-to-date as they otherwise might.

As for what the proper format to store internal data is, trying using MuseScore on a large score and you'll understand the importance of efficient representation. We try to eek out everything we can but still struggle. MuseScore 3 will get smarter about layout so performance will be helped by that, but the lesson is, a program as large and complex as MuseScore - and that is used to create large and complex scores - can't afford to take performance lightly. True, we could be even lazier and say, since we're not using this information now, let's be inefficient about how we store it, but that's not good programming practice either.

Reply

• Apr 18, 2018 - 19:08

Yeah, I do agree with your performance arguments. I was considering functionality more than implementation efficiency. In this case, I'd be surprised if the choice of implementation for mode makes a big enough difference in performance to be significant, but of course I haven't measured it.

> True, we could be even lazier and say, since we're not using this information now, let's be inefficient about how we store it, but that's not good programming practice either.

On the contrary, I would call that laziness good practice: defer your implementation decisions till as late as possible so you have the maximum amount of information when you make them. (Storing things inefficiently enough to make a difference in performance, of course, is not good practice, so there I do agree.)

Reply

• Jun 6, 2018 - 17:55

I just ran across a related bug, which I'm not sure if I should mention here or file as a separate bug. As mentioned above, MusicXML export understands minor in the MSCX file—except in one case (that I know of). If the key signature is A minor, and it is in measure 1 of the music, then minor doesn't get written to the MusicXML.

See the attached file https://musescore.org/sites/musescore.org/files/2018-06/minor_key_expor… and its MusicXML conversion at https://musescore.org/sites/musescore.org/files/2018-06/minor_key_expor…. Note that the A minor key signature in measure 3 gets converted to MusicXML correctly, whereas the one in measure 1 gets converted as C major.

Attachment Size
minor_key_export.xml 3.66 KB

Reply

• Jun 6, 2018 - 19:10

It's sort of a bug, but also sort of a consequence of how we represent key signatures. If a piece has no flats or sharps, we actually don't include a key signature at all. This leads to a number of problems - not having to do with MusicXML, but internally - and some day I hope we revisit this decision. But for now, it means there is nothing to hang the mode attribute on.

Reply

• Jun 6, 2018 - 19:29

Wait, what? I thought that neutral key and C major were represented differently internally in MuseScore (I know we talked about that earlier with regard to transposition).

But even if I do add an appropriate element to the MSCX file (as I did in the test file I posted), if it's in measure 1, it ignores the.

Reply

• Jun 6, 2018 - 20:18

They are indeed. Neutral key (atonal key signature) is represented in the score. But C major / A minor is not, not at the beginning of the score anyhow.

You can add an explicit C major / A minor key signature to the MSCX file, but it gets stripped out after reading it. There probably are backdoor ways of forcing it to remain - like maybe have a dummy staff that is transposed, or a have a part that has different transposition than the score - and maybe it will work today, but I wouldn't count on it.

Reply

• Jun 6, 2018 - 20:35

Yeah, I thought about looking for a back door, but it seems way too hackish for what I need to do with this project.

Intuitively I would have expected neutral key not to be represented and C major to be represented as 0 accidentals. But that's an implementation detail.

Reply

Do you still have an unanswered question? Please log in first to post your question.