Showing score in sync with external audio file playback

• Dec 12, 2021 - 19:14

Hi,
I saw some info about MuseScore 4 will be able to do show score in sync with external audio file play back. (I am explicitly avoiding term "playback score", as it typically associated with producing some sound. I am talking about "visual playback", showing score to player).
I remember that new architecture of Musescore player would allow for this feature - synchronized show of score and playback of external audio file.
I wonder what is the state of this feature.
I am looking to do some coding on it, can you give some info - in what directions to start looking in code.
Also, would be nice to know in advance, if feature will be accepted and merged into code.

I am thinking of integrated system:
1) May be extend Musescore file format to allow to store any audio format, like MP3 and company.
2) Some way to detect beats in audio file
3) Some way to produce/edit sync info, based on beats from audio file on one side, and bars and measures in score
4) Player of audio file (inside Musescore, of course)
5) Synchronized show of score, based on sync info from 3)

I do understand that can be way off with Musescore developer group opinion on product direction. Would be nice to hear how this feature set will blend into planned Musescore 4.

Thank you in advance for reading!

Sergey

Comments

• Dec 14, 2021 - 09:09

> "Do you know, is it possible to implement something like this with plugin?"
Perhaps, but probably not easily.

> "I.e. can plugin control speed how score is being showing up."
No

> "I am pretty sure plugin will let me to play some external audio file."
Not without launching some system-specific process calls.

Plugins can only do whatever a user can already do manually and not a lot more.

Reply

• Dec 14, 2021 - 13:28

Thanks for answering!
That is very interesting information...
I had a (apparently wrong) impression that new playback system in Musescore will be "advanced", and allow to implement stuff like show score, for every musician in a band, based on sync from centralized source - director score, may be even controlled by a wand in hand of director.
Most likely I was just dreaming :-)
Anyway, really, it is not possible to control playback system?? I have read explicit statements about re-writing playback subsystem in Musescore 4, to allow for more functionality. That actually made me very excited, as it opens a way to a very interesting functionality, like I just described above.

Reply

• Dec 14, 2021 - 15:35

The improved playback engine for MS4 will indeed be a lot more advanced, allowing for VST usage and much more articulation/expression interpretations.
That's the type of functionality that is targeted, making the default playback a lot more musical because the engine will get better at understanding a bunch of score markings it currently ignores (or only supports very minimally, such as legato).

Director score synchronization is a feature request, but most often made against the mobile closed source player companion apps of the score sharing platform musescore.com; those apps aren't support here on .org. It makes a bit more sense for the mobile apps to get that kind of feature (although I haven't seen any evidence that it'd be on their roadmap) as it is rather unlikely that your musicians will be running the full fledged notation editor software (as that is not supported on mobile devices).

Reply

• Dec 14, 2021 - 20:39

Yes, everything you said makes sense.
Requesting new feature from closed source software will not produce any results.
Speaking of mobile platform, Microsoft Surface technically a mobile platform, and it can run full-fledged Musescore. I do start seeing people running around with Surface, though probably it is less than 1%. So to say, even feature is implemented in notation editor, it would be still usable (with mention of Surface requirement).
That raises a question - how Musescore project is being run - it is really a community, or company?
If I will dig into Musescore code, will find a way to control playback from outside, would be my PR accepted into codebase?
I do not want to start doing anything until I will be sure that my code will made it into master.

Reply

• Dec 14, 2021 - 23:25

The desktop notation software project is managed by both; there are community reviewers and contributors but the final PR merging and release build packaging is done by the in-house team.

There is never a guarantee that a new feature PR will be included into the project, but you are welcome to join the developers discord chat channel to try and get a feeling for potential inclusion and approach discussion. You'll want to at least contact @Peter Jonas in there: he's the "community manager". Which means he's the bridge person that came from the contributing community and now works with the internal team to synchronize internal and community efforts.

Reply

• Dec 15, 2021 - 23:02

Thanks for explanations!
I will try to propose adding feature to Musescore playback - link to external sync server.
As soon as sync server is external to Musescore, there is quite a lot of options can be implemented.
That can be used to play score by a band, each musician will get individual playback of his part, under direction of director with wand.
Of course, it will be able to play in sync external audio file as well.
I think less specific sync feature would have more chances to be accepted.
What is developer discord channel - is it a forum "Technology preview"?

Reply

Do you still have an unanswered question? Please log in first to post your question.