GSoC 2016: Improve the synthesizers of MuseScore
Hello everyone,
I'm glad I got selected to work for MuseScore during GSoC 2016! I like to improve the two included synthesizers in MuseScore - namely fluidsynth (soundfont based) and zerberus (sfz based).
I would like to start with fluidsynth and add the feature of a crescendo/decrescendo during a note and legato play for wind and bowed instruments.
Zerberus should be able to handle looping of samples. (Right now there is no loop support at all) Furthermore having adequate ADSR for amplification seems very important. If there is more time I will also implement more OP-Codes.
My Fork of MuseScore on github: https://github.com/hpfmn/MuseScore
My Blog for GSoC 2016: https://gsoc2016.johanneswegener.de/
Comments
Welcome to MuseScore Johannes! You're taking on a big job so good luck and never hesitate to ask for help!
Yes welcome Johannes.
Please feel free to ask me about any MIDI issues you are not sure about.
In fact FluidSynth already supports crescendo/decrescendo. All that is required is to work out a way of sending the required string of Controller #11 messages from MuseScore to the synth.
Legato play is not going to be possible without modification to the soundfont. Specifically a set of samples needs to be produced with the attack section modified. Are you planning to implement this by velocity switching? Or program change? Unfortunately the soundfont 2.01 specification does not allow for controller switching, so it will have to be one of those two. Again, feel free to tap my knowledge.
In reply to Yes welcome Johannes. Please by ChurchOrganist
Thank you for your willingness to help! I appreciate that very much - especially since I got the impression that you have quite some knowledge in these topics!
From just looking at the fluidsynth source I would have guessed that CC 11 should be supported - I just was told that MuseScore's fluidsynth is behind the upstream one and I wasn't sure if it was just the Midi Controller not getting to fluidsynth or if it simply lacks that feature. I know (and tested) that upstream fluidsynth supports CC11 (which is also in the SF 2.01 spec).
I think I'll have to familiarize myself a little more with the code that gets midi from the outside world (aka jackmidi) ;) - I think most controllers get dropped in there.
I think sending midi messages during a note would be the first good thing to do (which should likely be done in rendermidi.cpp).
About legato I thought we could "just" change the parameters of the corresponding generators (attack/sample start) - either by NRPN (which is not so nice - I wrote a little python script for SF's NRPNs: https://github.com/hpfmn/pysfnrpn) or implementing CC 68 (legato footswitch) into fluidsynth. That would be an addition to the SF spec - but I'm not sure if that really is important for us?
Changing the soundfont might be an easy solution - but than it wouldn't be GM compatible anymore.
Please let me hear your thoughts!
In reply to Thank you for your by hpfmn
Hmmm - NRPNs - that may be a possibility - but they're very messy to use. I always preferred to use SysEx strings for such stuff. If, of course there is a way of reading the loop start point from the soundfont, that could work quite well, as you just tell the player to play the sample from that point.
It's insanely easy in SFZ as the opcodes are there to do that very thing, but I would need to look at the SF 2.01 spec again to see whether it is standard or whether you needed to write an extension. Controller #68 sounds a sensible way of doing this, and it may already be in the SF spec, but you may have to be prepared to hack the FLuidSynth code to implement it.
The other way, which occurred to me the other day, is to use the portamento controller (cc#65) with a fast portamento time which alters the pitch of the note without retriggering the sample.
In actual fact the default soundfont is only GM compatible in a retrospective manner - it always was a GM2 soundfont, and we have since added more new sounds using the GM2 Bank Select system.
Although, in fact using Velocity splits to implement a legato layer doesn't necessarily negate GM compatibility.
HTH
In reply to Yes welcome Johannes. Please by ChurchOrganist
If have some news about legato - have a look: https://gsoc2016.johanneswegener.de/2016/04/26/progress-on-fluidsynth-f…
Maybe you can give me some of your thoughts?
Regards,
Johannes
In reply to If have some news about by hpfmn
I'm sorry I can't give you any technical analysis (I'm reading rendermidi.cc and I can understand things but not the whole picture), but I would like just to praise and congratulate for you taking this effort, Johannes. We all know Musescore is primarily oriented to notation, that the focus should be adding more features and correcting bugs, but this possibility of having CC 11 implemented is a really BIG advancement, in my opinion, because many of us don't use external interfaces , just Musescore, and having volume changes implemented over single notes is definitely a wonderful thing for Musescore playback system.
I would add the capability to brass instruments too, and would like to give opinion on the frequency of the messages issue you mention in your web page. Perhaps following the same implementation as shown in Score::updateHairpin() (ie, incremental ticks) or...like the Bend implementation?
Well, again I'm sorry because I'm not familiar with the code/structs, this comment is to emphasize that this will be a wonderful, wonderful thing, if it could be implemented some day.
In reply to If have some news about by hpfmn
sorry I meant crescendo not legato ... sorry for that mistake ;)
Hi,
we decided that it might be a good idea to have to blog actually on musescore.org - so there is it: https://musescore.org/en/user/527826/blog
With the first post ported over from my wordpress!
This should make it easier to follow+comment for the musescore community (meaning you!)
In reply to Hi, we decided that it might by hpfmn
I posted a quite big blog post! And I'm desperate ;) for you feedback. So I'll leave a link here:
https://musescore.org/en/user/527826/blog/2016/05/19/status-and-gsoc-pl…
"If there is more time I will also implement more OP-Codes."
This.
I'd love to see as many of the .sfz op-codes implemented as feasible. Part of developing sound fonts in the .sfz format is to utilize the new enhancements these op-code afford. It seems counterproductive to a degree to go "just this far" with an implementation such as this and no farther. I realize there may be some insurmountable technical issues, but still, having as much compliance with the .sfz spec would be a win-win in my book.
Oh, and looping! Yes!
Looking forward to your work on this and I wish you the greatest success.
Cheers.
In reply to "If there is more time I will by jhouck
Hi,
of course having a complete set of opcodes would be awesome. But I don't think there is enough time to implement all of them during GSoC. So I'm just wondering what would be most important and hence I should start with these ones!
In reply to Hi, of course having a by hpfmn
Looping would be my first choice to be sure. Then ADSR envelope generators and LFO op-codes because both of these control pitch and amplitude control for notes. If time allows, any op-codes from sf2 that have sfz equivalents that haven't been implemented. In my opinion, for playback of traditional sampled instruments at least, I'm not sure there is much more needed.
Cheers.