New Instruments.xml file for 2.0

• Jan 4, 2013 - 20:25

I have now finished phase 1 of the task of making this MusicXML 3.0 compatible - getting a MusicXML id tag into each of our current Instrument.xml definitions, and am about to embark on phase 2 which is collating this information back into the MuseScore Instruments.xml file, together with including the rest of the MusicXML 3.0 instruments.

It is becoming increasingly clear that using the present two layer parsing system is going to mean an incredibly confusing and unwieldy instrument list in the Create Score dialogue, and needs to be replaced with something else.

I have had a look at some of the code which parses the exisitng file, and it would not be too difficult to implement a 3 layer system for 2.0 - just a question of adding a FOR loop to the exisiting nest. What is not clear, however, is at what level the More button kicks.

What I propose is that we make a completely new 3 layer system consisting of InstrumentCategory>InstrumentGroup>Instrument

It would work something along the lines of:

<InstrumentCategory id="Orchestral">
<InstrumentGroup id="Woodwind">
<Instrument id="Piccolo">
</Instrument>
<Instrument id="Flute">
</Instrument>
<Instrument id="Oboe">
</Instrument>
<Instrument id="Clarinet">
</Instrument>
<Instrument id="Bassoon">
</Instrument>
</InstrumentGroup>

<InstrumentGroup id="Brass">
<Instrument id="Trumpet">
</Instrument>
<Instrument id="Horn">
</Instrument>
<Instrument id="Trombone">
</Instrument>
<Instrument id="Tuba">
</Instrument>
</InstrumentGroup>

<InstrumentGroup id="Plucked">
<Instrument id="Guitar">
</Instrument>
<Instrument id="Harp">
</Instrument>
</InstrumentGroup>

<InstrumentGroup id="Tuned Percussion">
<Instrument id="Glockenspiel">
</Instrument>
<Instrument id="Xylophone">
</Instrument>
<Instrument id="Marimba">
</Instrument>
</InstrumentGroup>

<InstrumentGroup id="Untuned Percussion">
<Instrument id="Snare Drum">
</Instrument>
<Instrument id="Triangle">
</Instrument>
<Instrument id="Cymbals">
</Instrument>
<Instrument id="Bass Drum">
</Instrument>
</InstrumentGroup>

<InstrumentGroup id="Keyboards">
</InstrumentGroup>

<InstrumentGroup id="Strings">
</InstrumentGroup>

<InstrumentGroup id="Vocal">
</InstrumentGroup>

</InstrumentCategory>
<!-- End of Orch -->

A more complete example attached.

You would be able to click a more button in each Instrument group to be able to see the instruments in that group with the <extended> tag added.

This would mean that a basic list of the more common instruments would appear for each group, with the option of seeing all the instruments in that group if required.

The addition of a search box would also be useful for users wanting a particular instrument without having to trawl through the lists.

Comments?

Attachment Size
NewInstruments.xml 2.63 KB

Comments

• Jan 5, 2013 - 17:47

A very welcome step as far as I am concerned - thanks for the hard work that must have gone into this!

Some questions:

- I see "trumpet" listed as an instrument id under orchestral. Is that meant to to include Bb trumept only, so that you'd need to expand the rest of the Brass listing to see C and other trumpet options? I'd still rather see a way of expanding *just* the trumpet listing. But really, the biggest issue with the current system in this regard is that hitting the button to list all instruments causes you to lose your place in the list. If this were fixed, I wouldn't mind having to fully expand the Brass listing just to get C trumpets to show. The bottom line: from the moment I see "Trumpet" listed and decide that means Bb but I need C, I want it to only be a couple more clicks. I had proposed a whole new pane in the dialog where I could just override transposition (or other parameters), but that didn't seem a popular suggestion.

- Can instruments exist in more than one category? Eg, trumpets are used in both orchestral and jazz. Would be nice if I didn't have to go hunting. Search helps, but it would be very cool if expandfing the Jazz category showed me everything I would likely need so I could just click.

Reply

• Jan 5, 2013 - 18:01

This 'one instrument in more than one category' was my initial though too. That would call for some method to refer to another entry, to avoid duplication (like ragen, but also translation).

And I'd like to see some kind of tree to walk, like expanding a subtree by clicking on a +, which then turns into a -, clicking on that folds it again. We have that already, but only one level deep although this can and should have as many levels as needed and not any fixed amount.
And yes, the 'show more' should not throw you back to the start

Reply

• Jan 5, 2013 - 19:54

My intention would be to show instruments in more than one category, so Trumpet would appear in Orchestral, in Jazz or any other category it should be inclued in.

I also agree about being able to open up a list of trumpets to choose from.

The only problem about this is that I don't yet understand how MuseScore stores the instruments internally.

There is a data structure known as a linked list which enables the same array entry to appear in more than one place, and in my prior programming days this would have been how I handled it - ie you only have one Trumpet definition stored in the array, but pointers to that definition can be available in different sections of a menu.

Once I can find out what the current data structures are, I will be able to assess whether it will be possible to implement this for 2.0, or whether we will have to wait for the next rewrite.

And yes, tree menu structure would be very useful.

Thanks for your thoughts JoJo & Marc - keep them coming.

Reply

• Jan 7, 2013 - 15:32

Well I have begun adding instrument definitions.

It has now become clear that if we want a clear instrument list there needs to be four levels to the menu:-

InstrumentCategory
   InstrumentSection
         InstrumentGroup
               Instrument

I have identified the following categories as needed:-

  • Orchestral
  • Rock & Pop
  • Sound Effects
  • Ethnic
  • Early Music
  • Military

Are there any that I have missed??

These each split into the following Sections (as needed)

  • Woodwind
  • Brass
  • Untuned Percussion
  • Tuned Percussion
  • Keyboards
  • Vocal
  • Plucked
  • Strings

Are we happy that this is conventional score order???

The sections then split into the relevant groups of instruments - so Woodwind will split into Flute and Reed, Brass will split into Trumpets, Trombones and Horns etc

Although this will mean a degree of duplication, it will make it easier to select instruments for the genre you are working with, and will enable the uncommon instruments for an instrument group to be revealed by clicking a more button.

As far as I can tell this doesn't have severe implications for the coders - the current menu is built by means of a couple of nested FOR loops, and increasing those to four simply mean adding 4 lines of code.

Provided Werner and the rest of the coding team are happy to implement it for 2.0, I can see no reason why it should not go ahead, provided I can get the new file structure finished before the 2.0 launch date. This shouldn't be a problem although the more arcane instruments listed in the MusicXML 3.0 spec may well be missing and will be added later.

I just wish I could program in C++

Reply

• Jan 7, 2013 - 16:22

Can't we avoid any duplication, but using some kind references to another instrument definition?

We already have duplicates, e.g. violins-1 and violins-2, and bothe are almost complte duplicates of vilolin.
So violons-1 would refer to violin (e.g. "instrument refid=violin") , inherit all its settings and optionally just overwrite program vallue in the default channel, there would be no need to duplicate all the rest, like clef and range, etc.
violins-2's purpose is beyond me, but if needed, it could just be a reference without any overwrite

We could have an "acoustic Guitar" in Rock+Pop that just refers to "Guitar" in Ochestral and a 'Guitar' in Rock+Pop that refers to 'el. Guitar' in Rock+Pop

Reply

• Jan 7, 2013 - 16:33

The way I see it working is for there to be one instrument definition for each instrument, referenced to in each section by its MuseScore ID.

I haven't quite worked out how this can be implemented in the Instruments.xml file, but I think maybe the best solution would be to list all the instrument definitions at the end of the file with just id pointers in the Menu structure itself. This is pretty much how MuseScore works with the file at present - the instrument definition isn't read into MuseScore until the instrument is added to the score.

Provided the entire file is read into RAM during MuseScore initialising, there will be little impact on speed doing this. The file is unlikely to end up really large - the exisitng file is 192K, so I can't see it occupying more than 1MB - a signifcant amount in my programming days, but a fleabite with today's Ram sizes.

The alternative is to have two files - one with the menu structure, and one with the instrument definitions.

For ease of maintenance I would prefer one file.

Reply

• Jan 7, 2013 - 16:38

Hmm.

[EDIT: this and Michael's most recent response crossed, so some of this may be moot, but it's another way of looking at it]

With three levels of hierarchy below, I wonder now if the new top level is really needed. That is, what is the advantage at this point of having "orchestral" separate from "rock/pop" or "military"? Seems this is the source of the duplication. If the rest of the list underneath were going to be flat, then I could see that top level being enough of a convenience to be worth the duplication. Now, I'm not quite so sure.

And I'm also concerned about the possibility fhat it will in some cases turn out to be an *inconvenience* as people struggle to imagine which of these genre-based categories to go looking in when writing a piece that is not genre-specific - or is in a genre not covered. For instance, as things stand, if I were to want to write a jazz big band score, looking at that top level list, I'm kind of expecting I'd need to go looking for trumpets and trombone under orchestral, guitars, piano, bass, drums under rock pop, and hope that saxophone were to be found somewhere along the way. Sure, we could add an explicit "Jazz" top level category (or add that to Rock/Pop), but what about the person writing oh, say, music for a broadway-type show? Or - as I suspect is quite common among some MuseScore users - ad hoc combinations of instruments (eg, accordion, oboe, shakuhachi, electric guitar, saw wave, and lightning). Alll this does is make it harder to find instruments if your piece doesnt fit neatly into one of those predefined categories, plus it adds otherwise unnecessary duplication to the list.

On the other hand, I could see an extension of thus that would involve slightly more programming work but would allow this same basic idea to become much more useful, I think.

What if:

- The main portion instruments.xml organized the instruments without respect to the top level (that is, all woodwinds together in one place, not split up into orchesteal, rock/pop, and military with overlap). More or less just as now, but with that extra lower level of hierarchy - woodwinds / saxophones / Eb alto saxophone.

- There was a *separate* portion of instruments.xml that provided conveninece groupings such as "orchestral" et al. These conveinece groups would not reproduce the definitions of the instruments, but would merely contain references to them (by name or id or something). I suppose these conveinece groupings could also be broken down hierarchically into woodwids et al, but that might not be necessary or even helpful

- In the GUI, you'd see a top level hierarchy like this:

+ Orchestral
+ Rock/Pop
+ Jazz
...
+ All

Where "All" might be expanded by default (and I lised it last so the user could see right off the bat that there also exist convience categories).

The advantages of this approach would be that it would remove duplication, that it would be easier for the user not sure what genres to go looking in for the instruments he is using, and - perhaps most significantly - it would allow instruments.xml to be more easily customized with additional convenience groups that didn't require copying and pasting the whole instru,ent definition and creating yet more duplication.

Reply

• Jan 7, 2013 - 17:04

Now that is an excellent idea Marc

So the main list would be in a 3 level tree, and then the genre lists underneath.

That would be easy to maintain too, anyone wishing to add a genre would merely need to add it to the end of the file and then add the instrument IDs.

The file would now look like this:-

<InstrumentSection id="blahblah">
   <InstrumentGroup id="foo">
      <Instrument id="bah">
      </Instrument>
   </InstrumentGroup>
</InstrumentSection>

each one repeated

Then at the end of the file:-

<InstrumentGenre id="foo">
    <Instrument id="foo">
    </Instrument>
    <Instrument id="bah">
    </Instrument>
</InstrumentGenre>
<InstrumentGenre id="bah">
    <Instrument id="foo">
    </Instrument>
    <Instrument id="bah">
    </Instrument>
</InstrumentGenre>

I'd like to hear from Werner or Lasconic before I get too far with this to have input from the coder's point of view.

Reply

• Jan 7, 2013 - 16:54

I don't understand some of it (probably just me), but non-pitch percussion can be differently tuned, so I'm not sure 'non-tuned' would be the right term. That said, I'm not sure 'non-pitch' is either.

Reply

• Jan 14, 2013 - 21:43

I'm jumping into the discussion.

I also like the suggestion by Mark to have a list of all instruments sorted alphabetically, together with genre categories. It should be easy for newcomers to quickly find the instruments they want without having to thing about genres. That said I also think it is important having a search box to quickly find the instrument you want. As a matter of fact I'm implementing this feature as we speak for the "old" instrument list. It will probably be finished tomorrow and a pull request will be added to github. This search could easily be extended to the new layout. As you pointed out it is just a matter of introducing another loop.

Reply

• Jan 14, 2013 - 23:36

Believe me you do not want the instruments sorted alphabetically - that was one of the headaches I experienced with the MusicXML 3 list.

The current MuseScore default of having them in Orchestral score order is far better as it keeps families of instruments together.

Listing them alphabetically would mean that Oboe and Bassoon would be miles apart from each other.

I could see an argument for listing them alphabetically within Instrument Families, but that would definitely also seem counter intuitive to me.

The addition of a search box to that will be great :)

Reply

• Jan 15, 2013 - 07:21

Do you know how other programs like Sibelius or Finale does this? I still favor alphabetical sorting within the "All" group. Wouldn't that be the most intuitive? Within the other groups orchestral order is fine. Then MuseScore would offers three ways of finding the instruments you want.
1. You know what instruments you want and the genre so you look in the specific group.
2. You are not sure about which genre you music belongs to or you are mixing many instruments, so you choose the "All" instruments group. Then you get an alphabetically sorted list of all the instruments so it is easy to browse. If this group is also sorted by orchestral order I see no reason for having it.
3. Else you search for the instrument

Reply

• Jan 15, 2013 - 14:47

I've never seen a program provide a purely alphabetic listing of instruments, and would find it pretty awkward. A composer normally thinks of instruments by function more than name, and in groups of "related" instruments, where the notion of what's "related" has nothing the do with what letter they start with. Any system that didn't put piccolos near flutes, or oboes near clarinets, or trumpets near cornets, or trombones near baritone horns, would be incredibly foreign and awkward to most musicians. And much hard to navigate. Consider, when creating a score, we don't order and group the instruments alphabetically - we do so functionally. Immediately after selecting "oboe" in an orchestra score, we move on to clarinet. Wouldn't it much easier if they were adjacent rather than nowhere near each other?

You say you would only want to see this within the specific predefined genre groupings, but why oenaliz people who write cross-genre music? Just because I might want to mix some orchestral and some ethnic instruments does't mean I don't want oboes near clarinets. I agree that it shouldn't be necessary to think in terms of *genre* when selecting instruments, but thinking in terms of *function* should continue to be just as natural as it is now - we don't want to make writing cross-genre music harder by pretending brass instruments aren't related and mak,ing people go hunting for them rather than finding them together.

If a program provides only a single flat list - no grouping into brass, woodwinds, guitars, etc - then I guess alphabetic order would be the ly reasonable way to find things. But functional grouping is a much more logical way of presenting them. So I would expect the "All" listing to continue to break down by brass versus woodwinds, and as proposed hear, also break down the brass et al into further subgroupings like trumpets, horns, etc.

Of ocurse, with the current proposal, nothing would stop you from creating your own custom alphabetic "genre" grouping if that really suited you.

Reply

• Jan 15, 2013 - 14:54

I do see you point. And I'm not a professional composer so I do not have that much experience setting up scores with many instruments.

The reason I proposed having the "all" group sorted alphabetically was to have en easy way to browser all the instruments (are if you are scrolling the list after e.g. marimba it is way easier to locate if you have alphabetical sorting). But maybe that isn't needed. Or the search will do the job.

But what I don't see is the need for the all group if the sorting is exactly the same as the groups above it.

In what cases would a user open the "all" group if he knows it has the same sorting as the other groups? He would probably then have to scroll much longer down in the list that just opening a group.

Reply

• Jan 15, 2013 - 15:19

I *do* like the search box idea, and think it would be quite useful in case like where you you want a score for a single instrument where you don't know which functional group it fits within. If you know you can find "marimba" under "pitched percussion", I guarantee you'll find it faster (one click, no scrolling since the list of pitched percussion instruments isn't very long) within a functionally-grouped hierarchy that in a flat list that you have to scroll a long ways through to find. But if you don't know where to start, then indeed, a search box is a good way to go. Doesn't penalize those who *do* know where to find marimba.

As for why having the All listng at all, I think you are misunderstanding the different types of groupings. The All listing would contimue to be the main listing as far as I am concerned, and it would continue to be organized more or less as now - by instrument family (brass versus woodwind versus percussion versus stringed, etc). The new "genre" listings would be totally different. They would list instruments by the genre of music they are typically used in. That has nothing to do with whether they are brass or woodwind or whatever. So the "orchestral" listing would include some (but not all) woodwinds, some (but not all brass), some (but not all) percussion, etc - just the instruments typically used in orchestral music. It's a convenience for the benefit of people writing within a specific genre, collecting all the instruments they might need within a single place rather than scattering them among the different families. So *if* you are writing music that happens to fit within one of the handful of predefined genres, you can use the genre convenience listing to find all your instruments easily. But the All listing is still useful when writing music that does *not* fit neatly within a single genre - or perhaps, a genre not accounted for in the predefined listings.

Said another way: the All listing groups instruments by *family*, whereas the new conveneinece listings group them by *genre* - totally unrelated concepts. Within a given genre listing, the instruments would presumably be listed in conventional score order for that genre, which would basically have the effect of grouping them by family for the most part, but perhaps not exactly. For example, a genre listing for "pit orchestra" might have violin, viola, and cello, toward the top of the score with the winds, but "bass" at the bottom with the rhythm section. And wven to the extent that listing instruments in score order means they do happen to be grouped by family, that doesn't necessarily mean we actually include a hierarchical structure. A flat listing in score order would probably make sense for most genres, except perhaps "symphonic orchestra" where the list might be a bit too long for comfort.

Reply

• Jan 15, 2013 - 15:29

An interesting point, the inclusion of "genre" functionality shouldn't make it harder not to use a genre.

As an aside, note there can easily be a difference between the way it is good and easy to set up inputs (in this case instrument.xml) and how their contents are presented to the program user. Of course "order" then has to be specified in the data.

At the moment the order in the file specifies the order on screen. It is easy enough to reorder the display alphabetically because that's an order a computer knows about. If we had the instruments in the file ordered alphabetically and wanted them in "standard" orchestral order (for example), there would have to be something in the data that allowed the program to order them correctly.

That is a slightly silly example, but the opposite isn't necesaarily so daft. Would someone writing jazz band music always want the list in orchestral order?

Are we getting too involved in this issue? Does it matter that much? Does the new search function make it matter less?

Reply

• Jan 15, 2013 - 16:16

As an alphabetic sort is the sort of thing programmers used to do when learning their trade, I think it is safe to say that providing this facility would not be too taxing :)

The order the current file is in is the most intuitive form for most composers as they think in families of instruments, so if I wanted to use a capped reed instrument in my music I would go to that grouping of the Woodwind section.

The ability to sort in alphabetical order should the user desire it should not be an insurmountable problem, though you may have to wait a few seconds while the computer resorts the list.

It is, however, something I suspect not many musicians would use, as they would be looking for a particular sound rather than a particular instrument, and having them grouped as families as at present is a distinct advantage in this scenario.

Reply

• Jan 15, 2013 - 19:17

As far as I am concerned, either a button to produce an alphabetic sort on demand or a search box would solve the problem nicely. I don't know that both are needed, but I guess it wouldn't hurt. The alphabetic sort would be easier to design and code for, I would think but if the search box is already implemented, so much the better.

Reply

• Feb 23, 2013 - 21:21

If for "search" you mean "filter" I think that would benefit nicely; key one or more letters in an input box at the top of the list and the list narrows down to those items that have a match for the string you entered. Possibly a match on the beginning of the instrument name, so that "trump" would match Trumpet Bb, Trumpet C, etc.

The alphabetic sort would be easier to design and code for ... FYI programmers today have access to GUI controls that take care of all that coding for you. You just have to build a list and pass it to the code that calls the GUI element. I assume the toolkit used to develop Musescore is no exception.

Reply

• Jan 15, 2013 - 19:26

Having different orderings for different genre listings is exactly one of the thing I see the current proposal allowing for very naturally/easily. The "orchestra" genre listing would contain just the common orchestral instruments in score order; the "jazz" genre listing would do the same for jazz, and indeed, the order may well differ even when the instruments themselves overlap. Most obviously, bass goes above drums in any jazz score, but would typically go below in an orchestral score. So having the instruments.xml order be used directly in the dialog makes perfect sense to me. The order of instruments within the various families of the "all" grouping would matter a bit less than it might now. For instamce, I am constantly annoyed at how far I have to scroll down the woodwind list to find the saxophones - by far the most common woodwinds in the jazz world. They are used pretty seldom in orchestral music, and I'm not convicned they are placed properly within the woodwins list even for scores that do use saxohpones, but get the order right for the various genre listings and I won't care much about the order within the All grouping. I'd expect instruments to be broken up by family, and listing generally high to low probably makes sense, but if the instruents were alphabetic *within families* in the All list, that wouldn't break my heart, either. After all, is there really a standard for whether recorders go above saxophones, or where bagpipes go?

Reply

• Jan 15, 2013 - 11:35

To pick up the "dropdown box selection" point, this isn't to replace the current tree representation, it is to switch it between various types.

The dropdown box could have the list of "genres", of which "orchestral" is the only one currently available but you're discussing having others.

Selecting, for example, "orchestral" in the dropdown box would make the instrument display look the way it currently does. Selecting "jazz" would make the display show jazz instruments.
The in-program editor would need to allow the user to mark instruments as part of a genre, and perhaps even a heirarchy in that genre (just thinking out loud).

Reply

• Jan 15, 2013 - 19:31

Not sure what you mean about "orchestra" being the only one "currently" available. There are *no* genre listings currently available if by current you mean, 1.2 or a y recent nightly build of 2.0 as far as I know. there only the equivalent of the "All" listing, which has been and remains broken up by *familiesi*, not by *genre* . The genre listings like "orchestra" would be an entirely new thing. Whether they are presented alongside All in a tree hierarchy as we've been (mostly) discussing or whether there was a master dropdown that controlled the current view and the it was hierarchical tree below that doesn't matter much to me; either way it's the same amount of real estate and the same number of clicks to navigate, I think. ActuLly, I think having just one hierarchy that includes All and the genres as top level items has slight advantage that you could potentially have multiple genres open at once, thus reducing by one the number of clicks necessary to move between genre listings. But I can't see people writing mixed-genre music wanting to use the genre listings that much anyhow.

Reply

• Apr 27, 2013 - 16:35

I thought you guys would like to know that Phase 2 of this process is now complete - I submitted the final pull request for making the old Instruments.xml file XML 3 compatible about 10 minutes ago.

The next phase is for me to work with Mark Stanton on creating a new instrument structure, which will be populated from the old instrument file.

Reply

• May 14, 2013 - 23:42

Except that for Contrabass/Double Bass, the *default* sound is arco, and you need an explicit pizz to get that sound. Whereas for Acoustic Bass, it's the other way around. Orchestral scores will have the former expectation and require arco by default, jazz scores the latter. So, having separate instruments was the most logical way to make that happen, without requiring half the world to insert invisible arco or pizz markings just to get playback right.

Reply

• May 24, 2013 - 13:16

Hang drum is a pitched percussion instrument, developed in teh early 2000's and produced by PANArt Hangbau AG. Although it's a relatively new instrument and quite unknown, it has a lot of potential. I wouldn't blame musescore officials if they don't aquire it (it's expensive as hell and extremely hard to find) but I believe it deserves a position along with the rest of the instruments. Anyway, heres more info:
http://en.wikipedia.org/wiki/Hang_%28instrument%29

http://www.youtube.com/watch?v=Z_DhNLCyVss

http://www.youtube.com/watch?v=dWY6U2171qY
Cheers.

Reply

• May 24, 2013 - 15:23

I guess the question then is, what would you expect as far as support goes? MuseScore uses off-the shelf soundfonts. Since the instrument is so new, I rather doubt there are many soundfonts that include samples for it, so nothing MuseScore did would cause it to playback as expected. At best, you'd get the name to appear in the list, and then settle for some other sound. And you can already get that effect just by adding that other instrument instead and editing the name to say "hang drum". Or, if you do have access to a soundfont that includes this sound, you can install that soundfont yourself and use it already - no changes to MuseScore required.

Anyhow, not saying it isn't worth adding to the list. Just pointing out that if you're thinking that doing this would magically cause it to playback, you'd likely be disappointed.

Reply

• May 25, 2013 - 09:39

First of all it is not a drum according to the Wikipedia article, and is not classified as such in the MusicXML 3 sound list.

It is called simply a hang and the plural is hanghang

In order to enter it as an instrument into Musescore we will need more information about it.

It seems to me to fit best in the Tuned Percussion group?

What is the range of the instrument? And are there different amateur and professional ranges?

What kind of notation staff does it use, and what clef? Does it use a tab system or standard notation?

Is it a transposing instrument? If so what is the transposition interval? NB This can be octave transposition as in Glockenspiel

Finally, which GM sound do you wish to represent it?

Reply

Do you still have an unanswered question? Please log in first to post your question.