A plugin for syncing musescore with VLC for film scoring
To set up, go to settings in VLC and enable "HTTP web interface"
Then click "show all" and go to Interface > Main interfaces > Lua > and set password as "Test". You can set it to something else by changing the plugin code.
enter a value in the jump to time section and click "Go" to set Musescore and VLC both to that time. "Play/Pause" starts and stops both apps together from wherever they're currently positioned.
One issue - it has to set Musescore to an existing object, so it can be slightly off in musescore. I use a hidden tempo staff with a bunch of small rests so it always has something to grab on to at the right time. See screenshots. "triangle" is a tempo track.
The tempo staff should be the top staff, so the plugin goes there first.
UPDATE: New version, listed as vlc-sync-v12.2.qml.
When a staff is titled "TimeSeek", it will now automatically make all the rests into 64ths, so Musescore has an object to seek to. To force this, you can click "Update Map". It works also when the staff is hidden, so no need to see the ugly 64th note staff.
Alternately, you can just click "Go" and it will update.
Also included: graffesmusic's delay timer, although set to 25ms by default, as cfirwin pointed out that Musescore's play engine plays 25ms earlier than VLC.
| Attachment | Size |
|---|---|
| vlc-sync-v9.qml | 25.87 KB |
| Skärmavbild 1.png | 208.83 KB |
| Skärmavbild 2.png | 303.33 KB |
| Skärmavbild 3.png | 315.92 KB |
| Skärmavbild 4.png | 98.53 KB |
| vlc-sync-v12.2.qml | 35.06 KB |
| Skärmavbild v12.2.png | 515.96 KB |
Comments
Built for 4.4, works on 4.5 beta.
In reply to Built for 4.4, works on 4.5… by L'Moose
I will be putting this through its paces. Ironically, this showed up on the same day that the very robust Jack Transport PR was abandoned without plans for transition to another PR or rebuilding.
I see a number of issues with this, but it technically works. It seems to me that a solution that is connected to M4s midi time code output would be the best solution. In this way, items on the staff would not factor in the selection of time, but rather 'frames' would determine the sync points. Midi time code can be expressed outside of the Jack server (LTC, MTC, etc.). I don't know what VLC can do in the way of sync with midi time code or if VLC allows sync to frames/milliseconds, etc. A popular video player for sync that is cross platform is Xjadeo. It is strictly for video (no audio) but it has a variety of midi time code sync formats. This is the video player that Ardour (digital audio workstation) uses for video playback and sync. If both Audacity and M4 were made to sync in this way, then you would have a pretty powerful cross platform software suite for media composition.
In reply to I will be putting this… by cfirwin3
"seems to me that a solution that is connected to M4s midi time code output would be the best solution. In this way, items on the staff would not factor in the selection of time, but rather 'frames' would determine the sync points"
For sure. I couldn't find a way to do that. If anyone knows a way, I'm interested in implementing it.
In reply to "seems to me that a solution… by L'Moose
Would there be a way to create a drop down or second layer "offset" field that would allow for 1-9 tenths of a second addition (maybe subtraction) to the M4 starting point? Granted, this would not work in the negative (subtraction) when starting at the beginning of the score, as M4 would have no negative territory to place objects. However, in M4 one could create a calculation of advanced placement with an empty measure that would then align measure 2, beat 1 as the starting point for the picture (or any measure, for that matter). This would give the user the ability to create any kind of static offset that they wish in an effort to combat the delay of the play engine. Likewise, it would give the user "nudge" capabilities that are typical to other scoring features like this.
So I envision this:
The main field of the plugin has all of the settings that you have now. A an additional "time offset" popup, drop down, or field would allow the user to enter a value in seconds and tenths of a second: 00.0 (or maybe even up to hundredths of a second: 00.00)
Assuming that the field only allows for positive numbers, the plugin would automatically add this amount to M4s starting point indicated in the main seek field. It's a simple math solution that gives the user control. With positive numbers only, it would require that the user add an extra measure/tempo at the start, but it would also eliminate the need for video editing to acquire sync. With 64th rests or higher at almost any tempo, M4 should be able to calculate to the nearest point in a division of a second (up to .9 or .99).
Does that make sense?
The user would then derive all entered "go" points from the video timeline, ignoring the M4 timeline. The user can delete whatever buffer space is in the score before exporting, or align that extra margin in a video editor however they wish. This seems doable, given how the plugin works.
I have attached a mock-up of what it might look like.
In reply to Would there be a way to… by cfirwin3
I am working on a delay timer - start the vlc playback x ms after musescore.
![delay.png]( "delay.png")
@L'Moose
Do you allow contributions to your plugin?
In reply to I am working on a delay… by graffesmusic
If you improve it, no reason why not. Go for it.
In reply to I am working on a delay… by graffesmusic
That's better (cleaner).
Let me get my hands on it if you get it implemented!
In reply to That's better (cleaner). by cfirwin3
First test.
It is a bit difficult to evaluate if this works good enough.
I installed the Time addon https://addons.videolan.org/p/1154032/ , allowing onscreen timestamp in ms, but this does not seem to be very accurate neither.
So i added a delay field (default 200ms). There is a timer delaying with x ms the command to start vlc play, after musescore starts playing.
Only go+play button has the delay (or not if delay = 000). Play/pause does not have the delay.
Can also start at 00:00.
Shows up in MS as 'VLC Sync Delay' to differentiate from original.
Could you test if this can be working or if my idea is flawed?
Still needs more work, debug still shows some errors.
let me know how it works.
In reply to First test. It is a bit… by graffesmusic
Everything works as advertised. Obviously only the go+play can have the delay otherwise the play button will keep compounding delays with each stop and start. But I did find that multiple stops and starts from the regular button (after having used go+play) got the sound out of sync (probably because M4 is dealing with objects greater than milliseconds in distance while VLC is dealing in real time). So in practice, one should seek and start with go+play fairly regularly during the composition session to maintain the integrity of sync.
I wonder if there is a way to read VLCs clock to create a new start point after stopping (eg: when stop is pressed, the value of VLC is copied into the next start location -under the hood- and then the offset is applied to that value as a sort of back end go+play but only when regular play is pressed to start)? That was long, but hopefully the logic makes sense. This function could just take the whole second value from VLC from where it is stopped (so if stopped at 35.23... the new start is just 35). This value then could have the user entered offset applied to it and then both buttons would work all the time. There would be insignificant (micron level) skips when stopping and starting, but they wouldn't be perceived. Perhaps this might require the use of a dedicated start and dedicated stop button. In this way, there may only need to be a "go" button that would enter the location field the same way as the stop for the play button to apply the offset.
So we have the following logic:
1. "Jump Field" (where location is entered)
2. "Go Button" (puts the play head on the jump indication -AND- informs the Start Button of a starting place when pressed)
3. "Delay Field" (where delay is entered)
4. "Start Button" (plays from where either the Go Button or Stop Button indicates and always applies the Delay Field number when pressed. Go Button data always overrides Stop Button data in the Start Button location)
5. "Stop Button" (stops playback and records VLC whole second stopping point. This is then reported to the Start Button to use at the next start... unless overridden by the Go Button).
Let me know if I need to explain further. And again, I don't know what VLC is able to convey in terms of time locations when pausing.
In reply to Everything works as… by cfirwin3
So the status of vlc can be queried by sending a http command,
e.g. curl -u :Test -X POST http://localhost:8080/requests/status.xml
with time related results:
3 => position in the video, in seconds
17 => total length of video, in seconds
0.20007564127445 => position of the video as a fraction of the total video length. The position is a value between 0 and 1. A position of approx 0.200 means the video is approximately 20% through its total duration.
I don't think this is usable in the context of the plugin.
Is the out-of-sync situation after multiple stops and starts something already present in the original plugin or did i introduce it?
I think it is unavoidable with this solution, since there is not real sync.
The construction of the http request and sending that request to localhost takes some time, making the vlc starting a little later at every stop/start. Perhaps we could try to reset the time at every stop/start.
But, the only real solutions are (IMHO)
-using MTC (not present in MuseScore) or similar
-using a VST: if MuseScore could send ProcessContext.kPlaying ; ProcessContext.tempo ; ProcessContext.projectTimeSamples ; ProcessContext.projectTimeMusic to the VST.
See https://github.com/musescore/MuseScore/pull/26663#issuecomment-27356351…
The author of VidPlayVST confirmed to me that his plugin would work in MuseScore.
Perhaps this is not that complicated to achieve.
-jack, of course
In reply to So the status of vlc can be… by graffesmusic
VST cannot be the solution. It is not a cross platform method (Musescore Linux does not support VST and there has not been serious, timely or stable work, that I'm aware of, to make VST truly cross platform.)
I mean VST could be a solution... but I (and some others) would be TICKED if it were THE solution... especially after being lectured to about how Jack doesn't serve everybody equally.
How about this:
1. Can a plugin cause the M4 transport buttons to start/stop VLC?
2. Can the time data next to the M4 transport be used to automatically indicate the VLC starting position when play is pressed, or when a note is selected in the score?
2a. Can VLC seek to 1 tenth of a second as M4 indicates, or does it have to be a whole second?
3. If these can be achieved, then can there be the delay field that you have created to start VLC after a value entered in the field, but from the time indicated in the M4 transport location at the point of start?
I'm not sure that it's necessary to make M4 bend to the will of an external transport control (in the plugin). Instead it would be completely sufficient to have VLC be slave to the Start/Stop of M4 and the time data provided by M4 at a stopped moment.
Even if the outcome is approximate, Is this doable?
If not, then we essentially have the plugin that we have now, however I would recommend using the stopped time data from M4 as the indicator of location (if VLC allows selecting between seconds). In this way, when a user selects a note, VLC jumps (in pause) to the time indicated in the transport count. Likewise, when a user presses play, VLC jumps to the the time indicated in the transport count.
In reply to VST cannot be the solution. … by cfirwin3
Now there's a possible addition - making VLC stop time jump to the Musescore stop time, as calculated by the plugin, from Play/Pause.
So the pause option puts VLC to the same time as Musescore, so it can work better over longer sessions.
I wonder if one could have a check for if Musescore is playing (not the time output, just whether or not it's playing) and checking the start point against the plugin's internal tempo map and sending that to VLC simultaneously with playback start, making a more true sync.
In reply to Now there's a possible… by L'Moose
I don't think it would be critical to have a persistent check of time while playing. It would only matter at the point of start. So the plugin could collect data from that clock at start and/or stop and also when a note is selected. This data could trigger a VLC sync. But it only would work if VLC can seek at intervals between seconds.
The idea is to just allow the M4 transport clock to do the heavy lifting at all times. There would be no need to enter a start point in a field. Just click on a note or rest and the plugin would seek in VLC to where ever that number says. Alignment wouldn't be necessary while playing, just when starting.
You also wouldn't need to have a separate staff with "objects" at 64th+ values because the whole system would just treat M4 time values as master.
In reply to VST cannot be the solution. … by cfirwin3
VST cannot be the solution. It is not a cross platform method (Musescore Linux does not support VST and there has not been serious, timely or stable work, that I'm aware of, to make VST truly cross platform.)
I disagree here. VST3 is - potentially - completely cross platform, and the SDK can even be GPL. Any vst3 made with Juce can be compiled for Windows, Mac and Linux, in principle without code change.
Most recent VST's are made with Juce. But developers do not build Linux versions, because they do not want to support it. (perhaps for a good reason)
Have a look at my MuseScore Linux vst build:
https://github.com/diedeno/MuseScore/releases/tag/v4.5.0-lxvst
The only thing that is different from the official build, is enabling vst3 build for Linux. No single line of code has been added.
Since a couple of weeks, the missing Linux vst runloop is in the master.
https://github.com/musescore/MuseScore/pull/26424
Just needs testing (and maybe some bug fixing).
In contrast to Jack, this will actually happen.
In reply to VST cannot be the solution… by graffesmusic
Look... if they can get VST3 actually working (GUI shows up, plugins universally operate, etc.) in Linux, then that's great! I'm there!
My issue is that the pattern of development has been to implement working features in this area on Win/Mac and call that success. Then whatever doesn't work is left to an unknown and not guaranteed developer to deal with it, whenever. My experience with the VST PRs for Linux is that most of it simply isn't usable. GUI is not supported unless the VST is Juce. Implementation was highly suspect as of a few weeks ago when I last checked.
As you indicate here:
"Any vst3 made with Juce can be compiled for Windows, Mac and Linux, in principle without code change.
Most recent VST's are made with Juce. But developers do not build Linux versions, because they do not want to support it. (perhaps for a good reason)"
These are the reasons why that kind of dependence would not be a good way forward for solving this problem on Linux.
I will check out your build as soon as I get a chance! I'm not trying to diminish work that has been done, I am merely reacting to the very casual pattern of concern from the top admin with relationship to Linux features in this area.
So I'll rephrase: VST that doesn't work properly in Linux cannot be a solution for this feature. It would just be rubbing salt in a deep wound. If we decide to jump to VST right now for this... then I'm out because I probably won't be able to run it. Also if Jack is a no-go because it is an external system, then VST can equally get complicated with Linux when non Linux VSTs are dependent upon Wine and Yabridge.
It would be better IMO if this were a built-in feature on M4 or a cross platform plugin (like this project) or a cross platform transport sync system (however complex) like Jack. Jack is really the only think like Jack that works for all platforms, even though it seems that few windows users know how to use it and assume that it's not for them.
In reply to Look... if they can get VST3… by cfirwin3
Implementation was highly suspect
What exactly is suspect on my build?
In reply to Implementation was highly… by graffesmusic
Just the issue about dependency on Juce and the failure of GUI to load. I couldn't get anything to work (beyond superficial loading) the last time I looked at it (if it was your PR) a few weeks ago. It could see the VSTs, some would "load" but none would "work" (either lacking GUI or core function).
I have maintained that the Linux build might benefit more from an LV2 layer (as LV2 is a longer standing and more developed plugin format in Linux). But my guess is that feature is too Linux specific for most developers to tackle.
In reply to Implementation was highly… by graffesmusic
To be completely fair, almost all of the audio applications in Linux have 'some' issues with making VST plugins work consistently. Ardour and Carla won't load GUIs in some cases, they freeze or crash the program (as in Qtractor). So we have that going against us from the onset.
In reply to VST cannot be the solution… by graffesmusic
"Have a look at my MuseScore Linux vst build:
https://github.com/diedeno/MuseScore/releases/tag/v4.5.0-lxvst"
"Since a couple of weeks, the missing Linux vst runloop is in the master.
https://github.com/musescore/MuseScore/pull/26424"
I just fired up both of these and tried to open VSTs. They could be seen in the drop down. All of them did open. None of them had GUI so they weren't usable. Perhaps none of my VSTs are Juce? But they are the popular choices for my distro: like LSP plugins.
Now this was a test on an Ubuntu Studio install, which has had a lot of problems with MuseScore 4 since 4.3
I will circle back around and try it on an Xubuntu distro later.
In reply to "Have a look at my MuseScore… by cfirwin3
Same issue. No GUI in Xubuntu for a variety of VST3. LSP doesn't use Juce.
In reply to Same issue. No GUI in… by cfirwin3
Indeed, LSP does not use Juce. It also provides all Linux VST3's in a multi-plugin VST. This does not work in MuseScore. (and does not work in some others Daws as well, according 'the internet') . Solution: (answer from the developer):
Question: would it be possible to build Linux Vst3 plugins as separate plugins instead of a multi-plugin Vst3?
Yes. You can check out the desired plugin repository and build it like it is done with the lsp-plugins repository.
In other words: build it yourself. Like we used to do constantly in the old days.
After you have build the plugins yourself: you must switch to 'use new view'.
But really, do not feel obliged to try anything. I do not care.
In reply to LSP does not use Juce. It… by graffesmusic
Anyway, that exercise just to reiterate that I don't think VST is a good way to solve cross platform feature problems at this stage. There will most certainly be Linux specific problems. Please keep this video sync concept at the plugin level. I am begging.
In reply to Built for 4.4, works on 4.5… by L'Moose
The plugin causes the logfile to be flooded with:
.../Plugins/vlc-sync-v9.qml:137: Error: Cannot assign to non-existent property "__runs"
(does not affect functionality at first sight, but is nevertheless very annoying)
In reply to The plugin causes the… by graffesmusic
Do you think this is a result of the plugin failing to find objects at the designated time stamp? Or is it a persistent flooding when not in operation? Did you notice a problem with functionality popping up over time?
In reply to Do you think this is a… by cfirwin3
Probably not (i think). Perhaps the "__runs" property needs to be declared? ( non-existent property "__runs" )
Do you know how to prevent VLC to start the video automatically?
In reply to Probably not (i think)… by graffesmusic
Unfortunately, I don't think there is a setting for that. There is a setting to disable auto play from playlist. But this would require that you open a file by adding to play list first, and then select it to play. So when you load, just hit pause immediately and zero the timeline, if you need to. When using the plugin's GO feature, you won't really need to zero the timeline.
In reply to Built for 4.4, works on 4.5… by L'Moose
Is there a way to build in a .02 second delayed start for VLC when the plugin start is pressed? This would account for the static delay of MuseScore's play engine, and eliminate the need for a 2 second video and extra measure/tempo buffer:
https://youtu.be/xthWfiMnTkU?si=bvJLLtDelslE_cGk
In reply to Is there a way to build in a… by cfirwin3
If not possible with the plugin, perhaps a lua script inside Vlc could do the trick.
In reply to If not possible with the… by graffesmusic
I have tried the lua script approach , but it did not work out.
To prevent auto-start of videos, one can use
vlc --start-paused
In reply to I have tried the lua script… by graffesmusic
In what field do you put the start-paused command in VLC?
In reply to In what field do you put the… by cfirwin3
Start from cmd line. (or adapt the command behind the icons)
Not really tested, but at least first video does not start.
In reply to Built for 4.4, works on 4.5… by L'Moose
Here is a proper, complete video tutorial for the plugin and a suggested workflow:
https://youtu.be/trG2tiW5KRo?si=7Aw9s_4WylxBw3rv
In reply to Built for 4.4, works on 4.5… by L'Moose
Hi! I meticulously followed every step but get "Connection failed! Check VLC settings".
Any help please? (Using MuseScore 4.5.0)
In reply to Hi! I meticulously followed… by Michele Chiusi
The original instructions from the creator were as follows:
To set up, go to settings in VLC and enable "HTTP web interface"
Then click "show all" and go to Interface > Main interfaces > Lua > and set password as "Test". You can set it to something else by changing the plugin code.
It's possible that you might have to enter http or that the settings look a little different from mine (I'm on Linux). I believe that this was first tested on Windows by the creator.
In reply to The original instructions… by cfirwin3
I found the problem!
Apparently my VLC web interface doesn't work with "127.0.0.1" but it does with "localhost".
I changed the vlcHost parameter in the plugin code from 127.0.0.1 to localhost and now works fine 🤷♂
In reply to I found the problem!… by Michele Chiusi
ICYMI: I've made a version that gets the right time in Musescore more consistently, without the need for a playback staff.
https://musescore.org/en/node/376476
I found some 'bugs' in the plugin:
When only 1 (or no) tempo is specified, i have this in the logs:
11:46:55.652 | DEBUG | main_thread | buildTempoMap | Building tempo map...
11:46:55.652 | DEBUG | main_thread | buildTempoMap | Tempo map completed with 1 points
11:46:55.652 | DEBUG | main_thread | buildTempoMap | Total calculated duration: 0ms
When 2 tempos exist:
11:53:12.787 | DEBUG | main_thread | buildTempoMap | Building tempo map...
11:53:12.787 | DEBUG | main_thread | buildTempoMap | Tempo change at tick 3840: 1.5 → 1.333333 BPM (time: 5333.333333333334ms)
11:53:12.787 | DEBUG | main_thread | buildTempoMap | Tempo map completed with 2 points
11:53:12.787 | DEBUG | main_thread | buildTempoMap | Total calculated duration: 5333.333333333334ms
Whatever time is chosen if calculated tempo = 0ms:
11:55:21.190 | DEBUG | main_thread | findTickForTimeUsingTempoMap | Time 10s is beyond score end, extrapolating to tick 9600
11:55:21.190 | DEBUG | main_thread | positionMuseScoreAtTime | Target tick: 9600
11:55:21.190 | DEBUG | main_thread | positionMuseScoreAtTime | Actual tempo at position: 2 BPM
In case the plugin uses the cmd()
DEBUG | main_thread | positionMuseScoreAtTime | Failed to select element with cursor, trying cmd() approach
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: select-next-element
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: select-prev-element
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: score-position
2025-03-17T11:15:59.151 | DEBUG | main_thread | positionMuseScoreAtTime | Final selection status: failure
(see https://musescore.org/en/node/372762 )
In reply to When only 1 (or no) tempo is… by graffesmusic
So, does this plugin actually calculate time based off of tempo, or does it select the nearest event in the score based on a time provided by the play clock?
If it is the latter, then the plugin is merely a synchronized play start/stop control, right? But if it is the former, then those bugs might actually matter. I'm just trying to understand why a tempo map would be necessary at all, if the plugin is merely jumping to a playback clock time.
The bug re: "beyond score end" is probably not an issue, given the purpose of the plugin. Score and video lengths can and will differ.
In reply to So, does this plugin… by cfirwin3
It calculates time based off of tempo, since I couldn't access the playback clock time, hence the tempo map
In reply to It calculates time based off… by L'Moose
The problem with this
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: select-next-element
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: select-prev-element
2025-03-17T11:15:59.151 | WARN | main_thread | ActionsDispatcher::dispatch | not a registered action: score-position
is that those actions do not exist. It should be cmd("next-element") and cmd("prev-element"). Score-position does not exist at all.
V12.2 is great. Great, great progress. The delay work by graffesmusic saves a ton of work in video prep. And the Update Map feature is a lifesaver for long scores.
Thank you thank you!
Thanks a lot for your incredible work ! We film music composers missed a lot the video sync feature. It's a life changer !
Now, when we work on a long film and need to write different cues at different spots, that's useful to have an offset option for working on different parts of the film.
I've made some changes and it works perfectly for me. It allows users to shift the video's playback start time relative to the score, enabling synchronization with specific scenes (e.g., aligning the score's 00:00 at the video's 3:00 mark).
On my side the plugin version 2.0 v27 was more stable than 3.9.
Don't you think that opening a github page would be useful for that kind of community modifications ?
In reply to Thanks a lot for your… by Solrésol
The v3.9 has the time offset feature already. It's in the settings window. The user can enter HH:MM:SS in that field and the score's 00:00:00 will be set to that point in the video.
In reply to The v3.9 has the time offset… by cfirwin3
My bad ! Great !