New cascading instrument definition

• Aug 2, 2012 - 09:28

Werner recently committed a new architecture for the instrument list files. The instrument list files provide instrument name and characteristics for the new score wizard. The implementation in MuseScore 1.2 had several drawbacks, especially for internationalization. A translator had to translate the full instrument file. If an instrument characteristic such as the range had to be changed in the main file, the english one, then all the translated version had to be changed as well. The new scheme has been implemented to overcome this problem and Werner called it Cascading Instrument Definition in reference to CSS (Cascading Style Sheet).

How Cascading Instrument List works

There is one main default file in english, instruments.xml. This file is always loaded. The file is still very similar in structure to the one in MuseScore 1.2 described here. The main difference is that each instrument has an identifier “id”. Also note the different encoding of clefs. This file also contains informations about the way to play articulations, and could be extended further for legato etc...

The identifier is used as a key in the language specific file. Check the provided template
In this file, each instrument is identified with the id, and the translator just has to translate the name. This file can also be extended with specific instruments for the language or country using the same syntax that the main file. The second file can override all instrument properties. Only the grouping of instruments cannot be changed (no group or instrument can be deleted)

Both files location can be specified in Preferences -> Score. The default instrument list is loaded by default from MuseScore binary for best performance. (thus the path starting by :)

Future development

MusicXML 3.0 sound

Currently the ids are MuseScore specific. MusicXML 3.0 made a great effort to come up with a list of instruments, and a hierarchical id for them.
We could use them instead of our own ids when applicable. It would make it easier to export sound details to MusicXML, and maybe also import. But it involves a huge work of manual mapping or constructing manually a new instrument list based on the MusicXML 3.0 list. Anyone interested to help us? Comment here or join us on IRC (#musescore on freenode.net)

Load the language file of system language by default

The second instrument list file being language specific, it could be set by default to the system language. Non english speakers are often surprised when they open the new score wizard to find the list of instruments in english.

Localisation

We will need to organize the translation of the instrument long and short names. This translation could be done like it used to, by translating the XML file. But it could also be the occasion to make something better.

We had two ideas:

  • Integrate the translation into the translation server : http://translate.musescore.org. This will centralize the translation work for the software interface on this server. We will need some code to convert the XML file back and forth to a format the translate server can understand (po files).
  • Integrate the translation to MuseScore.org. We could create one page per instrument and ask translators to translate the page or add the translation on the page. Each instrument page could become a wiki page, with information on the instrument from the instrument list (range, clef, MIDI program) but also with external information from Wikipedia, link to soundfonts etc... This approach is interesting for SEO (Search engine optimisation): It will bring more people to MuseScore.org and then MuseScore. By diluting the translation process, we could end up with a slower translation process though. This approach needs some development to automatically create pages on musescore.org from the instruments.xml file, also delete/add new instrument if the file is updated. We also need a way to generate the per language instrument files out of the online pages.

Comments welcome!

Comments

• Aug 2, 2012 - 10:44

If you can let me know where to find the MusicXML 3 instrument list I will help with the Instrument file remapping.

Forget that - I've found it!

Reply

• Aug 2, 2012 - 12:13

That proposal sounds sensible.

I shan't have time to do anything to this before tomorrow, so if you'd care to come up with a definite data structure, I can conform to that.

My current thinking is to set the whole thing out on a spreadsheet with the MusicXML 3 instrument names in the first column, and the MuseScore names in the second - rows in between each instrument could then be populated with the extra data from the MuseScore Instruments.xml file.

I'll set that up as a Google Doc tomorrow morning with multi-user access so others can work on it too - you'll just need to let me have an email address so you can be added to the list of editing users.

This is likely to be a case of many hands make light work !

Reply

• Aug 3, 2012 - 13:49

I have identified a couple of issues already.

First:-
The MusicXML list is ordered differently from the MuseScore list.

The MusicXML list is ordered alphabetically according to instrument group

MUseScore's list is ordered according to the standard orchestral score order.

Do we wish to retain MuseScore's order?
Or adopt the MusicXML 3 order?

Second
The MusicXML instrument definition contains an id consisting of Instrument_group.instrument

Do we wish to adopt the MusicXML format?
Or retain the MuseScore system?

AM just about to start work adding the MuseScore instrument defs to the spreadsheet.

Update:
I've got about halfway through listing the MuseScore brass instruments next to the MusicXML 3 instrument list.

I have made a few modifications along the way as outlined in the Notes column.

There are already inconsistencies between the 2 formats appearing, but I hope that the way I have set it out will enable eventual easy integration of the two.

One thing that is becoming clear is that it is probably not going to be possible to use the MusicXML identifiers in MuseScore, not without drastic changes to the instrument list anyway, but it would be better that an extra field be introduced into the Instrument definition to hold the MusicXML instrument identifier.

Reply

• Aug 3, 2012 - 18:49

1. List order

Does the order of the XML file influence the order of the instruments in the Instruments dialogue box?

If no, I believe the point to be secondary (only maintainers of the file itself will notice its order).

If yes, my vote is for keeping current MuseScore order, which is the 'expected' order for any user familiar with scores (i.e. the majority of the users having any expectation at all about the instrument list).

2. Id format

Assuming the MXML ID will be purely internal (i.e. no final user will see a string like "brass.bugle.euphonium-bugle"), this point might be a consequence of other considerations (order of the instruments within the file, position of the ID in the Instrument tag, ease of reference, etc...).

As MXML Id's lack key info and lump variants of an instrument into a single Id (for instance, there is only one lute), keeping the MXLM Id's as such might be problematic, as Micheal noted above. But which would be the reason to keep the MXML format, if the Id's themselves are modified?

(if you like, I could take care of the viol section, if nobody else volunteers; but there are only 6 of them, not a big offer, I know...)

M.

Reply

• Aug 3, 2012 - 21:09

1 - The instruments.xml order is the one the user sees in the MuseScore UI. So I would not change it except if something looks wrong.

2- How I see it, we would replace the MuseScore id by the MusicXML one when available and keep the current one if not available, or change it to a hierarchical one for consistency AND we would add a new tag to store the MusicXMLid, empty or absent if no id is defined.

Reply

• Aug 4, 2012 - 11:07

Surely it would be simpler just to add the MusicXML id tag to the instrument definition and leave the MuseScore definitions alone?

It is usually easier to add a field to existing code rather than go through and change identifiers throughout the MuseScore source.

I have just thought of an issue whilst completing the brass section this morning.

We're OK with exporting to MusicXML as MuseScore will just export the MXML identifier from the list, but how about the import of MXML? There are some MXML instruments which do not exist in MuseSCore.

We will need to either develop a system for dealing with undefined instruments, or input a MuseScore default equivalent into the missing instruments - I have made a start on some of the more obvious - eg Sackbutt uses the Trombone defs, but there are things like Vuvuzela and didgeridoo which may need a complete new Instrument def in MuseScore.

@Miwarre - you now have editing access btw

Reply

• Aug 4, 2012 - 13:29

The brass section is now complete apart from a few MusicXML instruments which have left me stumped as to what to represent them with.

If there is a brass specialist around who can check the ranges that would be a good thing.

Incidentally, while we're in the process of altering the Instruments.xml file structure, can I suggest we add a Bank Select field to the Channel structure. this will enable the use of expanded sounds beyond the range of GM as outlined in our discussions in the Soundfont Forum: Beyond GM - standard for instrument assignments

Will be starting on the percussion section next.

Reply

• Aug 4, 2012 - 13:54

Let's first do the MuseScore -> MusicXML mapping. We could add new instrument in the list if needed.

Regarding "Beyond GM", adding Bank select in the instruments list would not be enough I guess. Bank select will have to be sent to the sequencer, the synthesizer will have to support GS or XG soundfont, etc...

Reply

• Aug 4, 2012 - 15:24

The SoundFont 2 specification already supports bank select messages as does Fluid, so presumably it would be a question of getting MuseScore to pass the messages on to the Fluid sequencer assuming that hasn't been hacked out in the MuseScore customisation. I don't think it can have been otherwise Drumkit selection wouldn't work.

Nothing needs to be done to the code immediately - MuseScore could just skip over that part of the instrument def, but making it part of the Instruments.xml file would make sense while it is being overhauled, and doing it now allows for much expansion of playback features in the future.

Reply

• Aug 6, 2012 - 09:03

Every instrument has at least one Channel definition. The Channel defines the setup for a midi channel and can contain an arbitrary midi initialization sequence. Usually only the midi "program" value is defined. Bank switching can be done with midi "controller" definitions (Example is unpitched percussion instruments which define controller 0 (bank select)).
You can also set initial values for volume, pan, reverb send and chorus send which are all interpreted by MuseScore.

Reply

• Aug 17, 2012 - 18:15

For the various sizes of zink, notes in the spreadsheet say:

"MuseScore is using the German name for this instrument. It may be better to change the name to ... Cornett".

My mother tongue is not English, but the Wikipedia article says: "The cornett, cornetto, or zink is an early wind instrument..."; so, zink seems potentially acceptable as an English term. It has in addition the great advantage to avoid any confusion with the cornet (just one 't' less).

So, I vote for keeping zink (or to use cornetto which is less similar to cornet).

M.

P.S.: I wonder why MXML puts it in the brass section...

Reply

• Aug 8, 2012 - 17:51

I've done the German translation meanwhile. It may not be perfect or complete, but is ls already better than the 1.2 version ever was.
See pull requests #33 (accepted meanwhile) and #34 (pending)

Reply

• Aug 9, 2012 - 13:34

Update: The untuned drum section from MuseScore has now been added to the Google Spreadsheet, together with some of the percussion effects. There are, however, a wide range of ethnic drums present in MusicXML 3.0 which are not in MuseScore.

Is there a percussion specialist who could look at these and give some advice as to how they would be added?

Incidentally Marching Percussion is currently missing from the new Instruments.xml file. As it is already part of 1.2 I am planning to add it over the next few days, and will invoke a "pull" via GitHub. I am also planning to split off that part of the TimGM6MbwithDL, and turn it into a separate SoundFont for use with 2.0's multiple soundfont ability.

In terms of future expansion it might be a good idea to add a SoundFont field to the Instrument definition for use by users wishing to use a number of soundfonts on their system for different instruments.

Are we planning to become MusicXML 3 .0 compatible in MuseScore 2.0 or is this planned for a future version??

Reply

• Aug 9, 2012 - 14:22

We plan to be MusicXML 3.0 compatible in MuseScore 2.0. It already the case in the nightly in fact. MusicXML is a selective encoding standard so if the instrument name is missing in export, the importer has to figure something out. So adding the sound attribute will a bonus, to be "more" compatible.

Reply

• Aug 14, 2012 - 06:32

Just a comment on the current draft. All the flat signs in the spreadsheet are currently listed as lowercase b's. These need to be replaced with
<symbol name="flat"/>

(This applies to the "longName", "shortName", and "description" tags)

Reply

• Aug 14, 2012 - 07:08

A note about "brass.posthorn": Mahler writes for Posthorn in Bb, Symphony No 3, Mov. 3 around rehearsal 14 (the sheet music uses the German letter B which means Bb in English, see PDF p. 50 of the following edition: http://imslp.org/wiki/Special:ImagefromIndex/109864

Videos of the "Post Horn Gallop" on the web seem to play in Eb or Ab (and appear to use natural herald trumpets instead of post horns). Unfortunately IMSLP didn't have a copy of the original sheet music.

Reply

• Aug 14, 2012 - 07:28

"brass.trumpet" should correspond with Bb trumpet (definitely not Eb trumpet)
"brass.trumpet.slide" should probably be Bb too. The key for slide trumpet was not standardized in the Renaissance. Modern trumpets are usually in Bb
"brass.tuba" should in the key of C

I noticed that the transpose tags are missing from the draft spread sheet. Is this intentional? Please note Bb tuba is not a transposing instrument, despite its name.

Reply

• Aug 14, 2012 - 08:52

The Eb trumpet was put in brass.trumpet because there is nowhere else for it to go, the other instruments from the trumpet family all having a section to themselves.

I will copy Bb Trumpet up there too.

If you would like editing rights David, let me have hyour email address.

Reply

• Aug 14, 2012 - 08:38

All the MuseScore instruments were copied verbatim from direct from the original Instruments.xml list which represents the flat sign with a lowercase b.

I was assuming that MuseScore would internally translate these into flat signs, but on checking in a recent nightly I see that is not the case.

Would it not be better to use the Unicode character??

Reply

• Aug 15, 2012 - 02:48

I use Wordpad quite a bit. Not that I'm opposed to installing something else, but I figure as as much a tinkerer as the next guy, so I still use Wordpad, I'll bet a lot of others do, too.

I just recently noticed that MusicXML export uses direct UTF characters where I expect something more high level that would read as ordinary charctaers in Wordpad (like "\flat" or whatever) as I first expected, or at least an escape code like &x01d6a; or something. I'm OK with that now, but was surprised by this behavior.

Reply

• Aug 15, 2012 - 08:31

"Neither Notepad nor Wordpad can deal with that Unicode sign"

Wrong! Both Notepad and Wordpad will display these signs provided you have a Unicode font set, such as Arial Unicode.

Of the editors I have access to both Editpad and TED Notepad have no problems, but PSPad does with the last stable version.

The only problem is entering them without continually having to fiddle with the Character Map.

TED Notepad seems to have no means at all, Editpad and PSPAd both have an internal character map, but again it is fiddly.

Both Notepad and Wordpad have a means whereby you type in the hexadecimal UTF code then press Alt+X, but then you lose out on XML syntax checking and structure highlighting.

Reply

• Aug 15, 2012 - 14:40

I don't know that Notepad gives a choice of fonts. Wordpad does; not sure if the default is customizable.

Anyhow, the main issue to me is editability. That's the main point of a text (as opposed to binary) format for things like this. I mean who cares if you can *read* a configuration file; the main reason you would normally look at it is to edit it. Neither of the methods of entering Unicode characters (via copy/paste from character map or entering numeric codes directly) are very friendly. But then, it's not like people will be editing these files on a regular basis.

Similarly, few people would be editing MusicXML often, which is why I have no real complaints with it being essentially read-only.

Reply

• Aug 16, 2012 - 16:34

It is in your list, but not filled with the MuseScore defition, as is none of the voices.
And I don't know what a "range of G3 or A3 to E5 or G5" means in terms of instruments.xml.

Hmm, seems like:
<aPitchRange>57-76</aPitchRange>
<pPitchRange>55-79</pPitchRange>

right? Would this mean G clef like Alto/soprano, which have similar range, or G8va clef like Tenor?

Reply

• Aug 16, 2012 - 20:52

The countertenor range is very similar to the Alto, in fact, strictly speaking it duplicates it, however in practice a counter tenor would be able to include the bottom end of the tenor range as well.

So I would put Amateur pitch range as G2 to E4 and professional as C2 to F4 where C3 is middle C. A really good professional might be able to squeak a G4, but you'restarting to get into male soprano ranges here.

In terms of MIDI notes that is 55-88 and 48-89.

BTW jojo I've given you editing access, but your notification email bounced with a "Domain name not found" error not sure why. Let me know if you have access problems.

Reply

• Aug 17, 2012 - 11:15

Ah if you double click in the cell then paste from instruments.xml it will include all the selection and automatically strips out the returns. You can then put them back in :(

Incidentally I have come across a very useful tool for entering any unicode characters.

You can find it here: http://www.law.net.ru/unik/index.htm this solves the problem of entering the Unicode flat sign ♭ (which I've just entered with a key combination created with this tool).

Reply

• Aug 28, 2012 - 11:41

I've finally finished untuned percussion - nearly sent me mental in the process, but now it is done :)

There are a huge number of discrepancies between MuseScore and MusicXML here - instruments not in MusicXML and in MuseScore and vice versa.

We could really do with a percussion specialist to look at this - any takers?

Gonna rest for a bit to let my head cool down then I will tackle keyboards.

Reply

• Aug 29, 2012 - 21:07

I think I did allmost all woodwinds and plucked strings (a few missing that are in MuseScore but not in MusicXML, will add them shorly) and all voices.
Brass seems complete
Guitars are added now too... as well as most keyboards. I left out Synthesizer and Organ

Reply

• Aug 30, 2012 - 08:58

That is a big help Jojo :)

Here are the conclusions we came to regarding the Instrument Defs and MusicXML 3.0 at the Hangout yesterday:-

Nicolas on what ID should be picked in the final instruments.xml:

We keep our id when it exists
We add musicxml id
If an instrument is defined in MusicXML but not in MuseScore we add it
If an instrument in MuseScore but not in MusicXML, we try to assign a close sound id if we can. If we can’t we assign nothing.

This will enable us to fill in the gaps effectively I think.

A new field is being added to the Instrument Def: MusicXML ID - can't show the syntax here as the forum thinks that arrow brackets hold a tag! But I will be starting to add these to the MuseScore side of the spreadsheet so you will see how it works :)

Reply

• Sep 15, 2012 - 09:45

I think we now have all the MuseScore instruments entered into the Google spreadsheet.

I have now begun entering the MusicXML ids into the MuseScore side of the spreadsheet - I have added the field after the Description field - if you'd like it in a different place eg immediately after the MuseScore instrument id then please say before I get much furether :)

I am about halfway through brass at the moment, but have had to take a break due to work commitments. Whilst entering these I have been adjusting and adding instrument defs to fill in the holes in the MuseScore list compared to MusicXML

If anyone else is able to assist with this - it would speed things along :) Editing access merely requires your email address adding to the list - thought it not a good idea to give access to anyone with the link - we have enough trouble with spammers as it is!

Reply

• Sep 27, 2012 - 21:27

I looked at the MusicXML 3.0 definition, and there are 2 harmonica type listed
...
sound id="wind.reed.harmonica"/
sound id="wind.reed.harmonica.bass"/
...

Is this possible to add more harmonica type: chordette and chord. If I understand the MusicXML correctly it should really be listed as with harmonica been a general class:

...
sound id="wind.reed.harmonica"/
sound id="wind.reed.harmonica.diatonic"/
sound id="wind.reed.harmonica.diatonic.A"/
sound id="wind.reed.harmonica.diatonic.C"/
sound id="wind.reed.harmonica.diatonic.D"/
sound id="wind.reed.harmonica.bass"/
sound id="wind.reed.harmonica.chordette"/
sound id="wind.reed.harmonica.chord"/
sound id="wind.reed.harmonica.chromatic"/
...

Am I correct?

Reply

• Sep 28, 2012 - 00:45

Yes you are correct, but additions to the MusicXML Spec are beyond our remit.

We can put the new instruments into the MuseScore list, but until the guys that administer MusicXML get their act together on this they will be exported into MusicXML as "wind.reed.harmonica"

If you can provide us with the necessary data to define them for MuseScore, they will go into 2.0 along with the rest of the definitions.

Regards
Michael

Reply

• Sep 28, 2012 - 14:54

I have questions,

Will you provide with edit permission?

I looked at the information to provide in column Musescore Instrument, but I am unfamiliar with them. Can you explain?

We have Harmonica tuned in C, G, D etc... Do I have to provide and entry for each one?

Will there be someone else to validate the entries I make?

Thanks,

Michel

Reply

• Sep 29, 2012 - 16:35

I'm not sure it's valuable to add an entry per Harmonica tuning. What would change for each except the name? Even if the ambitus is different, I'm not sure it's worth adding each one of them. The only reason I can see if they have a different transposition like a C and Bb trumpet but I think it's not the case. Am I wrong ?

Reply

• Sep 30, 2012 - 13:56

The reason for allocating an entry for each harmonica diatonic tuning is because certain notes need to be 'bent' when played.

A diatonic has 10 holes, with blow/draw action this gives 20 positions to play 3 octave. To reach the missing notes, we apply special blow/draw technique at specific hole position. Therefore a chord chart will show those 'bent' notes and it is specific for each tuning. Harmonica popularity is pushing company to innovate. A new harmonica diatonic is now available from Susuki where the use of overblow have been eliminated. An overblow is a technique very difficult to master. Therefore, elaborating a chord chart for that specific 'C' diatonic will be different than a current 'C'.

The concept of displaying harmonica notes on staff has been brought up a couple of times: http://musescore.org/en/node/9386. On that point, is there any progress?

In conclusion, my point is pertinent only with the perspective that we can produce a chord chart specific for harmonica. If this is the case, then we should have an entry for each tuning. Within each tuning, we may have another level of specificity.

Reply

• Sep 30, 2012 - 14:08

Edit access given Snowman :)

It looks as though we will need a range of harmonica definitions then.

The best plan would be to take the original harmonica definition and copy it, then make the necessary adjustments to names, ranges etc

I will then make sure it's compatible with the rest of the sheet in due course.

Thanks for doing this

Reply

• Sep 30, 2012 - 18:31

Hi,

I looked in the spreadsheet and there is no detailed description into wind.reed.harmonica.

Browsing through the spreadsheet, generally it's the same description that's been used. So let's pick one. Can you detailed what it means, because the spreadsheet does not have an area explaining what the properties with '?' means?
Is there more properties I should include?

Instrument id="harmonica-bass">
longName>Harmonica Bass
shortName>H. Bs.
description>Harmonica Bass
clef>C
?barlineSpan>1
?aPitchRange>52-81
?pPitchRange>52-83
?transposeDiatonic>-4
?transposeChromatic>-7
?Channel>
?program value="69"/>
?/Channel>
/Instrument>

May be we could add a line at the end of the spreadsheet detailing what every properties means?

Reply

Do you still have an unanswered question? Please log in first to post your question.