Management of big scores

• Jun 6, 2019 - 22:45

Hi everyone,

Musescore slows down when anaging big scores.
Did you ever think using several cores on the CPU or even cudas on GPU ?

Comments

• Jun 6, 2019 - 22:53

If you compare 2.3.2 to 3.1, you should find it's actually gotten a ton better. If you have a score where there is still a noticeable issue, please attach it so we can investigate further.

Reply

• Aug 17, 2019 - 05:52

I'm currently doing a 30-minute-score (I'll attach it once it's completed) and MuseScore apparently slows down. I agree it's much better since MuseScore 2, but in my understanding -- even if the score is big, my minor edits don't change anything I'm not editing, except for structural adjustments. Does the software really have to process a lot of things I'm not editing? If the software is not supposed to do so can we avoid it?

Reply

• Aug 17, 2019 - 14:42

If you like us to investigate and make improvements, please don't wait until you're finished to attach your score. Nthst just prevents us from getting a head start, and may mean we miss an an opportunity to act before the next update.

Most edit operations should indeed only cause MuseScore to process the measures that changed. But keep in mind that if you change one measure one a line, it affects the rest of that line exauee relative widths may be affected. And if it causes a measure to be pushed the next line, that affects the next line, and so on. So there can be "ripple effects" that can result in a page or more. Plus, some commands inherent affect the whole score (style change, etc).

In addition, continuous view has some unique challenges that result in some things always needing to be applied to the whole score - layout of elements normally done system by system, like lines.

Reply

• Aug 18, 2019 - 15:36

Dragging a page or any other form of scrolling doesn't involve any significant recalculations at all. More likely, you've simply run out of RAM and you are limited by your disk speed and your OS's swapping efficiency.

FWIW, on my system with 8 GB RAM, scrolling / dragging is absolutely instantaneous. A normal edit is also virtually instantaneous in page view but indeed lags noiceably in continuous view - about half a second per edit on average (Core i5 processor). Changes that affect the score are very slow as expected - several seconds each.

Reply

• Aug 19, 2019 - 09:07

Honestly I've switched several computers to run MuseScore 3 and they all have the same problem. A moment ago I ran MuseScore immediately after I started my computer, the RAM cannot run away so rapidly, can it? You said dragging didn't cause any recalculation, but the fact is dragging the 40-minute score is obviously slower than the smaller ones. The latter responded instantaneously, but the former, though didn't freeze, moved much slower than my mouse.

And about editing: yes, the first or second or even third edit in page view on a big score doesn't have an obvious delay, but after a few (like seven or eight) edits the entire screen freezes for about ten seconds, and then continue to be active, and frozen, and active...

I'm not an IT person, so my deduction can be wrong, but does MuseScore have some algorithm that continuously fresh up the screen every ms or μs? This is the only cause I can think of.

Reply

• Aug 19, 2019 - 15:39

One thing that is clear is that every edit causes some sort of relayout. In page view I never edit fast enough that MuseScore 3 freezes as you suggest, but I keep auto save turned off. My computer is so slow on save that some scores take more than 2 minutes to auto save so I just turned it off. I now start every new system with a ctrl+s to prevent myself from loosing data. In continuous view when the score gets a certain length, probably around 6000 measures total (which would be 200 measure for 30 instruments) I start seeing a noticeable slowdown and after adding another 50 - 100 measure to the score (1500-3000 more total) I would start seeing what you describe when I add items too fast. But as I said, this is only in continuous view, I never see this in page view.

I have an 826 measure 41 instrument score (that's almost 34,000 measures total) and edits are instantaneous in page view. When I entered the score in version 2, I had to break down the score into 4 parts and merge them together to make a single score or edits were so slow that my computer would pause after every edit and would freeze if I typed too fast.

Reply

• Aug 20, 2019 - 01:07

I don't know either, but it is not autosave nor memory, I've checked the system monitor and the available memory is about 3 GB and doesn't change significantly when inserting or editing a spacer, so it seems that memory isn't the culprit either. I think there is a problem in the algorithm deciding the scope of the relayout, which triggers, even in page view, a complete relayout instead of limiting to the current page. Probably it is not an obvious condition, so some people in some computers are experiencing it and others not. The fact that the developers cannot reproduce it is unfortunate because it reduces the chance of having it fixed.
The problem is roughly proportional to the number of notes, or probably notes + other symbols, rather than the number of measures or staves per system.
By the way, my laptop is also 7 years old, but I'm on Windows 7 home premium (may be this has anything to do?). It has an i7 processor with 8 cores, and it is really fast with other programs such as mathematical programs or graphical programs. I use sometimes Apophysis 7x, a fractal image program that is very computationally demanding and it works as fast as could be expected. That's why I feel frustrated when the software´s UI freezes.

Reply

• Aug 20, 2019 - 01:23

You may be right about the number of measures and notes some how affecting this. I just added a spacer to my largest score (840+ measures with 41 instruments) and got a very noticeable delay, about 4-5 seconds. I tried it with a score with 450+ measures and 15 instruments and there was a much shorter pause. I then tried it with a very short 6 instrument piece and 200 measures only 20 with notes and there was a barely perceptible pause.

This wouldn't be the first edit that did more than is necessary for the action that has been found. It just doesn't happen as often as the others and takes quite a few total measures to make a long enough pause to give it a second thought (my medium score has a total of about 6700 measures). Only people doing large symphonic scores would ever notice this.

Reply

• Aug 20, 2019 - 02:05

It is true that the base of the MuseScore users community are involved in piano, guitar, chamber, choral or band music. But a good performance at this (good responsiveness) could boost the adoption of MuseScore by composers self-publishing their own symphonic compositions, as well as many vocational or state orchestras. The local symphony orchestra in my city has been using Sibelius, but the new copyist has a knowledge of MuseScore and could adopt it if convinced that it is highly competitive with commercial software..

Reply

• Aug 20, 2019 - 02:25

I think the next release, that is supposed to let you change the names of instruments after an instrument change, will make MuseScore highly competitive, capable of doing almost anything the copyist would need to be able to do. There are a few glitches here and there, but I think the price of Sibelius is makes MuseScore the far better deal. As you've seen there is a community here willing to help find solutions.

Reply

• Aug 20, 2019 - 15:15

This feature is already in the nightly builds and the results are backwards compatible (scores created this way will look as expected in older versions too) so feel free to start using them. Just use Staff Properties to change the staff name normally and it will pay attention to which measure you clicked and make changes to the instrument in effect at that point.

Reply

• Aug 19, 2019 - 12:37

If the problem is one of RAM, then indeed larger scores would be more affected than smaller. And right after you start MuseScore the score is still probably being paged in from disk, so I would expect more of a delay than after you've been working with the score for a while. Because as I said, we really don't do any major calculations on drag. There is a "layout" and a "draw" and the layout is the expensive part, especially when it happens for the whole score. The draw is much faster and it is always just for the portion of the score actually in view, so it really shouldn't be much affected by score size. A tiny because it takes longer to decide if an element is in view or not for a big score, but that time should be practically imperceptible.

Anyhow, I couldn't reproduce any problem with dragging with your score on my machine, and yet other operations were slow just as I reported, so there really is something system-dependent here. I really do think RAM / swapping issues in your OS are the most likely explanation. Were the different computers all running the same OS? Anecdotal evidence on these forums suggests that somehow macOS is less efficient about this than Windows.

Reply

• Aug 20, 2019 - 12:14

I'm using Windows and I admit that desktop computers are much better than tablets about this. But we really shouldn't act like this is a minor problem just because some computers have a really good performance. I'm afraid a software, at least a music notation software, deserves the commendment of "high efficiency" if and only if it can run very smoothly on most computers.

I know there are similar softwares, like Adobe Premiere Pro, which requires a big RAM to run effectively, but it deals with large (probably over a gigabyte) video files. The so-called large scores which MuseScore deals with is usually no more than a megabyte. We can take Microsoft Word for a far-fetched analogy: do you often notice a serious lag when editing, say, a file of several hundred pages, of text, images, tables and many more? I'm afraid not, not as obvious as MuseScore is, when using a normal tablet computer.

The point is that the freedom and comfort the user feels shouldn't at all depend on the size of the file they're working on. There are very different MuseScorers in this community, some enjoy making solo scores while others, like me, often writes orchestral (multi-voice) pieces; Some prefer pithy works while others like immense ones. I don't think I feel very well if my commits are much restricted in speed than others'.

As I said, I'm not an IT person, but meanwhile I've done a handful of programs including tools and games, and my intuition is that the optimization of the running speed probably requires overall enhancement of the fundamental algorithmic logic, something like the workflow of buffer areas (or even the grammar of mscz/mscx files?).

Again, I'm not professional and until now I can barely understand what are the functions in MuseScore code doing, so you can ignore the previous paragraph. I'm really not questioning all your hard efforts contributed in MuseScore, you've done an amazing job and MuseScore has been my favourite notation software of all time. I just don't know how severe you consider this problem to be, because from your posted comments, I cannot see any hint of definite resolution of improving it. The most important thing I'm stressing is that if I were the OP and this were posted as an "issue", I would definitely mark it as "Critical".

Reply

• Aug 20, 2019 - 12:29

I'm not saying it's a minor problem for the people who experience it, but I am saying until we understand how to reproduce it, there isn't much we can do about it. So we rely on those who do experience it to do some detective work for us.

And to be clear - we have spent an enormous amount of time over the past few years improving the performance of MsueScore on large scores. That's why it works so well in most cases, where most edits are virtually instant regardless of the size of your score. This was absolutely not the case in MuseScore 2, and it wasn't true in 3.0 for continuous view. Only as a result of a lot of work did we get where we are today. But if some issue on your system is preventing you from seeing these benefits, we definitely want to understand why. We just can't investigate further until we understand how to reproduce the problem you are seeing.

Reply

• Aug 20, 2019 - 13:15

In a related thread I noted that I did some testing with spacers and discovered there is probably some bogus calculations being done under the hood. I have a score with 850+ measures and 40+ instruments (about 34,000 measures total). I entered a spacer and it took 4-5 seconds for MuseScore to respond. I don't have a timer on my computer telling me how long it takes so I count "one one thousand..." In contrast changing a note was instantaneous. I then added a spacer to a song with about 250 measures and 15 instruments (about 6800 measures) and it took about half as long for MuseScore to update. I then tried it on a score with 6 instruments and 200 measures (only 20 measures have notes) and there was a slightly noticeable delay from releasing the spacer to the time of the update. It was clearly slower than changing the note in the 850 measure piece. I saw no noticeable difference if I inserted the spacers at the start or the end of the pieces.

Reply

• Aug 20, 2019 - 13:21

Since right now my only theory is what you are seeing is RAM dependent, it would be good to verify that by finding a computer with plenty of RAM, verifying things work OK, then slowly start removing RAM (physically or virtually) and testing again. That sort of thing. Or bringing up Task Manager or some other resource monitoring tools and seeing if you can any insight into what is happening that way. I don't really know, to be honest.

You say you don't see scrolling in Word slow down on large files, and that may well be, but again, I don't see scrolling MuseScore slow down on large files either. So comparing those side by side could e useful. Still, understand that music layout is far more complex than text. And FWIW, photo editors are notoriously slow with large images and little RAM, even just for scrolling.

Reply

• Aug 20, 2019 - 14:36

I tried shooting a screen record but it seems to be too big to upload here... (it can be a problem with the website, because the file truly uploaded, I saw my web monitor reporting a 300KB/s upload, but after it went back to 0, nothing was really attached)

Anyway I used a computer with 8GB RAM. Before I opened the big score (already got a smaller score loaded) MuseScore's RAM usage was 170+ MB, but the moment I opened the big one it rushed onto 240+ MB and kept being that high (maybe this is one of the causes?). Several minutes later when I was dragging the pages, I believe it was the auto-save so the screen froze for 10+ seconds. Then every time I clicked save the screen froze. Adding notes and copy-and-pasting had only a small delay, but adding breaks or spacers can be very slow. Dragging and zooming in/out was merely acceptable, still much slower than the small scores, if I do one of them after some quick edits.

Reply

• Aug 20, 2019 - 15:29

I would upload video files somewhere else and post a link here if they seem relevant. But really, it's probably only relevant if you can show the difference in pattern between system with, say, 8 GB RAM compared to 4 GB or whatever.

The fact that save will lock the program for the duration of the write is known, we don't really need to see evidence of that to believe it. If this causes problems for you, probably best to disable autosave )but then be sure to save regularly yourself, at times when you don't mind the delay).

Reply

• Aug 20, 2019 - 17:31

I don't think the RAM theory is correct, since at least in my case I have 6 GB RAM and the task manager shows that even at the situations where the program freezes during several seconds there is plenty of free memory, even with other programs running, a condition that has proven not to affect significantly the performance problem we are discussing. Indeed, hundreds of times more free memory than the most conservative guess of memory usage by the program would indicate.
Asking people who are mere users for a detectivesque work they aren't in condition to perform doesn't seem the best way to figure out where is the problem. I think the investigation needs to be done at a debugging level, which requires working with the code, or using much sophisticated monitoring software. Most of us are just not able to face this endeavour.
You keep asking for scores where this happens but when uploaded you can't reproduce the problem. I propose the following experiment: load a large public-domain score (there are many sites where MIDI files of classical or romantic symphonic works can be downloaded; I have already uploaded some in this forum, i could upload one again if you need it). Then select all, copy and start appending copies at the end. I guess at some point you will start experiencing the problem. I could also try to prepare some progressive examples if that can help (I hope this overload doesn't crash my MuseScore...).
Another experiment would be to try to test the problem in a less powerful machine than the one you regularly use.
Please note that nothing I've said, here or in past comments, is meant to minimize the enormous effort and commitment that is evident in any single tiny feature of MuseScore, let alone the last improvements and the upcoming ones.

Reply

• Aug 20, 2019 - 17:49

If the RAM theory is not correct, I would encourage you to continue to investigate until you have a theory you can validate with data that we can then use to reproduce the problem.

Again, freezing during operations that trigger a full relayout - that is something we do fully understand. Full layout is known to be an expensive operation, it's far more complex than anything a word processor does so not surprising that they don't suffer (try a non-destrutive photo editing program and exporting the edits to a TIFF - now you have something comparable, and it's comparably slow). We solved this problem as well as we reasonably could by not doing a full relayout nearly as we did in MuseScore 2. No doubt a few cases remain, like adding spacers, where we apparently still do a full relayout unnecessarily. So I encourage anyone who has the incentive, ability, and time to work on this to try replacing the setLayoutAll() call with a triggerLayout() call in the Measure::drop() handler for spacers, and then testing to see if a new fix is required for #273766: Automatic Placement with "Hide empty staves". That would be a productive use of someone's time to be sure.

But to be clear: the only thing anyone has reported here that I don't understand and cannot reproduce and need others to actually investigate the cause of is the claim that in some cases,, even dragging the score is slow. This is the thing I specifically say should not be and that I cannot reproduce, even with the largest scores anyone has been able to provide me.

Reply

• Aug 21, 2019 - 04:29

I've always been thinking if the delay has to do with unnecessary layout, so it's a big step forward that you've found the cause :D

About dragging the pages, there's only one thing I want to know: do you think it's completely not understandable even though there's just a tiny bit slower, or it's normal for big scores to be slow but not understandable if it's too slow or even freezes?

Reply

• Aug 21, 2019 - 12:31

Again, it's not that we have "found" a cause here. It has been known for years that a full layout is slow with large scores That is why so much work went into implementing partial layout for MuseScore 3, so most operations can be fast. No one has ever claimed or suggested that full layout should be fast with a large score - we all know it won't be.

Once again, the only thing being questioned here is the claim of long delays on mere scrolling operations, which don't even do a partial layout they do no layout whatsoever. If there is a ten secord delay just dragging a page, this is something not expected at all, regardless of the size of the score. It is the one and only thing mentioned on this thread that we would need help in reproducing in order to understand.

Right now, once again, the only plausible explanation (other than autosave kicking in at that moment, which of course doesn't happen on every drag, and can be turned off anyhow) is the system has become critically low on RAM and the OS is spending lots of time swapping. In which case, MuseScore is hardly the only thing that would be affected.

Reply

• Aug 21, 2019 - 14:45

Big scores take RAM, that shouldn't be surprising. Again, unless your system is basically out of it because of whatever else is going on, or your OS is doing a terrible job of managing virtual memory, scrolling should still be pretty much instantaneous. So if you are still seeing a ten second delay scrolling the score you attached, what we still need it to understand what variable you can change that would fix it - adding more RAM, closing other processes, etc. Because right now you seem to be the only one who sees ten second scrolling delays.

Reply

• Aug 21, 2019 - 15:20

I see only the smallest perceptible delay when I drag large scores with several pages showing. I do see some of the delay reported when adding spacers, but my delay is about half of what fmiyara reported.

It seems the people who are seeing a long delay in dragging a score are using windows 7, but I could be wrong.

Reply

• Apr 17, 2023 - 20:39

Listen you guys , the program is CPU hungry, i run MS4 on a Thinkpad i5 6600U and 32GB of ram, my CPU jumps to 70% in most of the MS4 use , scrolling in continuous view is not a problem, but after the first playback of a section, returning and replay it, staters!!! Also when you select a note to edit it plays it back numerous times,it is very difficult to stand at times. Now I really like the sounds although a bit heavy, but the way they respond to score markings is desirable, please see what it can be done to fix this. PS the program doesn't take up ram at all.

Reply

• Apr 17, 2023 - 22:05

In order for someone to investigate, we'd need you to attach the score that is giving you issues, and give precise steps to reproduce the problem. note this thread is about an entirely different version of the program, so not really relevant here - best to start a new thread.

Reply

• Aug 20, 2019 - 16:23

I couldn't agree more with Howard_C. The example of Microsoft Word is perfect. I'm using a 2003 version and have written nearly 10 books using it, several of them with many figures and formulas, some with over 500 pages, and it barely lags any command, not even saving, which is smoothly performed in the background. This is the unfortunate reason why I couldn't so far adopt Libre Office, which I would love to: many LO commands lag even with an almost blank page.
The problem does exist, as has been repeatedly expressed by several heavy users, and downplaying its importance just because of failure to reproduce on a specific machine doesn't seem a good policy for future development.

Reply

• Aug 20, 2019 - 16:57

No one is "downplaying" anything, just stating the plain and simple unchangeable fact: we can't investigate or fix a problem we can't reproduce. So, we could use your help in coming up with the precise steps needed to duplicate the elements of an environment that would enable the problem to be observed.

Reply

• Aug 21, 2019 - 05:57

I'll try, but I haven't much expectations since surely you'll report that you do not observe what I observe (not your fault, of course). I've attached a version of Rossini's William Tell repeating three times the original material (which I had downloaded previously as a MIDI file and imported into MuseScore). It is a score with 1432 system- measures, 25 staves/system, i.e. 35800 measures in total, and a duration of 35 minutes.

These are my observations on page view:
1) When dragging the score at 100 % zoom I experience a slight lag, but by no means intolerable
2) When dragging at 25 % zoom I experience some lag, may be half a second to start moving; not one I would complain about.
3) When I insert a 3 sp staff spacer down, either at the begining, at the end or anywhere in the middle, regardless of being at 25 % or 100 % I get a delay of about 11 s, during which the UI freezes and no further action can be done.
4) Surprisingly, deleting or undoing is immediate
5) After 3), setting spacer height to 9 from the inspector takes another 11 s. Setting height to 10 takes about 21 s, half to respond to the "1" and half to respond to the "0". However, writing "10" elsewhere, copying and pasting into the inspector box causes the requestion action to take 11 s.
6) After changing size enough to push the next staff downwards, deleting the spacer or undoing is immediate
7) Inserting 3 spacers (different staves), requires 3 operations and about 35 s. Selecting the spacers and changing the height to 9 sp (i.e., enough to push the respective neighboring staves) takes only 11 s, i.e., the same as changing the height of only one. Undoing or redoing height change takes 11 s. However, selecting the three spacers and deleting is immediate. Undoing or redoing delete is immediate.
8) Repeating 7) with 15 spacers yields the same result: batch changing their heights takes the same 11 s as a single spacer, deleting is immediate and so is undoing and redoing delete.

On continuous view the observations are similar, with the exception that instead of 11 s in most operations it takes 14 s, and in the case of deleting and undoing/redoing delete it takes about 3 s instead ob being immediate. I'm tempted to say that passing from page view to continuous view implies an increase of 3 s in the delay or lag in most cases, but I can't tell if it is just a coincidence.

My system has an i7 processor, 6 GB RAM, 27 GB available in system disk, Windows 7 Home Premium. After opening MuseScore with this score the task manager indicates that MuseScore uses 572 MB of memory. There are 3.2 GB available. When perfoming the operations decribed above the memory usage doesn't grow significantly, the CPU column changes from 0 to about 12 for a while (not sure if it is a percentage) and when complted if falls again to 0.

If you think there are other tests I can do and report, please do tell me.

Attachment Size
Rossini Guillermo Tell x 3.mscz 469.82 KB

Reply

• Aug 21, 2019 - 12:25

I fear there is still some confusion about what I am saying. Almost nothing of what you just wrote about performance in the Rossini score surprises me in the least or is inconsistent with what I have been saying. Let me summarize:

1) full layout operations take time proportional to the size of the score
2) partial layout operations take time proportional to the size of the change
3) most operations require only a partial layout and thus are fast in both page and continuous view
4) layout is still slower in continuous view than page view for a number of reasons
5) a few operations require full layout (including save), which is why they are slow with large scores
6) scrolling should never be affected by score size at all unless your system is RAM-limited
7) scrolling speed is affected by the number of items to be drawn, so scrolling at smaller zoom factors will be slower than at larger zoom factors

If you consider these known facts and think about what you just described, it's all consistent. Inserting a spacer currently causes a full layout (something I didn't realize when the thread started but learned over the course of it). Previously I mentioned exactly where in the code this happens, how it would need to change to make it partial, and which bug was fixed by adding the full layout, so one can test if the bug remains fixed after changing to partial. Right now changing spacer properties also triggers a full layout, in Spacer::setProperty(). The same change there would also be effective but would also need testing to see if problems are introduced.

Again, though: none of this should be surprising, nor was an example needed to demonstrate any of this. The only thing said in this thread that I was questioning is the claim that for some particular score on some particular system, merely scrolling a large score caused delays of several seconds. This is the only thing that doesn't seem perfectly consistent with everything we know about how MuseScore works, the only thing that I still would need to understand specific conditions (aside from merely running out of RAM) that could cause it.

The fact that full layouts are slow is, once again, non-controversial. That's why we work to void them if at all possible, and convert full to partial where we discover opportunities (and @mike320 has pointed several we have fixed already). Spacers may turn out to be another.

Meanwhile, mitigating the effects of full layout by moving it to the background sounds like a nice idea. Almost certainly easier said than done, though. And again, the price paid may well be that scrolling then becomes slow as it cannot commence until the operation completely. Also, most likely a second edit still could not commence until the first completed, as we need the results of the first layout to have correct information to begin the second. I suspect the actual improvement in overall experience would not be as great as you might hope. Combined with the difficulty of the task, I am not overly optimistic about this approach. But I certainly would love to be proved wrong if someone were to undertake it!

Reply

• Aug 21, 2019 - 16:20

Marc, I cite your message yesterday (Aug 20, 2019 - 09:24): "I expect adding a spacer to require a full relayout in continuous view because it affects the staff spacing scorewide. Only in page view would I expect to see the time to be comparable to a single note edit."
So you seem to be implying there that in page view, which most of my observations above refer to, I should find almost no delay to insert a spacer. However I've shown with my example (on my system, I admit it) that the largest part of the delay, 11 s, is due to page view, and only 3 s can be attributed to the continuous view. From your comment I would expect the opposite. That's the origin of my confusion.
Personally I don't care too much about spacers in continuous view since I consider using them a bad practice (trying to format the music while writing or trascribing it seems to be a waste of time). But I'm most concerned about spacers in page view since it is the final step after inputting all musically meaningful symbols like notes, dynamics, etc., and, as not a minor usage detail, this process is the last one, usually when the deadline is very close :).
Anyway, nothing of this would be much of a problem if the automatic alignment to the bottom (as has been discussed in other thread) were implemented, since the spacer use would limit to very particular cases overcrowded with ledger lines, dynamics, trills, ottavas and the like, and would even be less necessary if autoplacement extended to these elements as well.

PD: There is also a problem in this forum's interface in the way it indents replies, which makes it very difficult to detect at a glance what previous comment is answering a given comment, that's why there may be some confusion regarding two main subjects here: dragging problems and spacer insertion and editing problems. EDIT: This comment should be indented with respect to yours but it is on the same vertical line.

Reply

• Aug 21, 2019 - 17:13

Right, and as I already acknowledged at least twice since that post, I was mistaken about adding a spacer being an operation that requires only a partial layout. See in particular https://musescore.org/en/node/290398#comment-940605, posted only a few hours after the message you cited. In that followup, I mention the exact location where the code controlling this, what change would need to be made, and gave a good clue about what would be needed to test.

What confused me, on the other hand, was your suggestion in https://musescore.org/en/node/290398#comment-940492 that "it's autosave or memory". That comments *specifically in response to the portion of the thread that was about scrolling. That lead me to believe you were seeing huge slowdowns in scrolling. And then you repeated this later in that same thread, saying "I don't think the RAM theory is correct". Once again, the "RAm theory" is about one thing and one thing only: a possible explanation for 10 second pauses just scrolling. It was proposed as an explanation for that and neever in any way whatsoever suggested as an explanation for anything else

So are you now saying you never actually saw any slowdown in scrolling - meaning my RAM theory is in fact most likely still plausible - and the only slow operation you are talking about involves spacers, which we all known trigger a full layout and hence there is no mystery whatsoever here? In which case, I would again strongly encourage someone to follow the leads I already provided. Unfortunately I am completely engrossed in other projects right now.

Reply

• Aug 21, 2019 - 19:53

Scrolling slowdown used to be a problem for me in previous versions. Now, if any, it is very slight, may be in cases like the example I posted less than half a second. Not quite a problem. May be a RAM problem, but if it is the case, why not a video card RAM memory, since those who commented that probem don't see it reflected in the system memory report from the task manager.
By the way, all this confusion arises from the particular way the forum interface works, where it is difficult to know what part of the thread one is answering. May be the maximum number of indents is set too low.

Reply

• Aug 21, 2019 - 19:56

Two more questions:
1) Is full relayout performed at page view in some cases? In such case, is the spacer insertion one of those cases?
2) Is indeed ever necessary a full relayout or it would be possible to perform a forward relayout? I mean, a relayout from the place where the edit has been done onwards, or, in some cases involving at most the previous page (for instance if replacing the first few measures content with rests or with very few notes, those measures could jump to the previous system, which could be on the previous page; only the last system on the last page could be affected, I think).
The forward relayout would be most useful since often problems arise when appending new notes since only rarely one goes back to edit something in the middle. I mean other problems appart from scrolling or adding spacers, such as notes being added that freeze sounding for some seconds and many others that I have had recently using 3.2, which forced me to split the score into two or more parts and seem directly related to score size.

Reply

• Aug 22, 2019 - 00:11

Both continuous and page view have the exact same distinction between full and partial relayout. Some commands require full relayout, most are optimized to only need partial, but it's the same distinction regardless of mode. So if adding a spacers causes a full relayout in continuous view, it does in page view too.

The only thing unique about continuous view is that it is one long system, so elements that are dependent on the system for layout - like lines and lyrics - will need to be laid out for the full score even when during partial lyout. This could probably be optimized further some day, which is something I pointed out earlier. But probably adding a spacer will continue to effectively require the whole score to be laid out in continuous view, because even if we do a "partial" layout, in this particular case that does mean the whole system, and the system is the score. Spacing of staves in continuous view is handled specially to avoid making things too slow, but it's not ideal right now since even lyrics don't automatically add space.

Right now there is no concept of a 'forward layout". Partial layout is forward layout, stopping when it detects no changes. The commands that require full layout are mostly ones like style changes that truly do affect the full score. So it's not clear why spacers are in that category.

Reply

• Aug 20, 2019 - 17:22

As you probably know, this is of great interest to me since I mostly work with large symphonic scores. I've grown accustomed to using page view for copying, but I still prefer continuous view for original compositions. I sent Marc a very large score for testing and we (mostly Marc) is looking at some possible causes of the long lag that happens when you add a spacer in page view. The point of my saying this is that I have a score where the slowdown can be tested and there is now a theory. This is a vital step in fixing any problem.

I have been where you are - quite frustrated with the program and Marc. Once we are able to take a step back and look at possibilities we can look for the cause of the problem and find a solution. I won't claim that the solution is known at this point and Marc does have other obligations, but I'll second that this is not unimportant.

Reply

• Aug 20, 2019 - 20:22

I can't honestly say that I am frustrated with the program or Marc. The program is very good and regarding Marc, I very much appreciate the way he takes the time to answer most messages with a thoughtful comment, even if sometimes we differ. Of course the program can be improved, otherwise no one would be working on new versions. My humble contribution is to try to report problems that can plague the workflow of the user and try to provide as far as I am able some possible ideas.
In my experience, sometimes the solution for a problem isn't to try to find where the problem is rooted, but to start over in a different direction.

Reply

• Aug 20, 2019 - 21:19

MuseScore 3 did start over in a different direction and we have a much faster score update than we did in version 2. With this being an open source project, there are few people on the payroll to do a bunch of testing and trouble shooting, so us users need to help find issues. It's a small price to pay and several users spend more time helping with this. Initially MuseScore 3 was terribly slow due to several bugs, but most of those have been worked out and we are where we are today as far as speed is concerned. As for continuous view, I think it's generally agreed the concept needs to be rethought so it can be further improved. Starting over is a very difficult thing to do in the middle of a release, so any starting over will likely wait until at least version 4 whenever or whatever that ends up being.

Reply

• Aug 20, 2019 - 21:28

I'm completely aware of that, I don't expect next month to have the issue solved, but things like single note dynamics, that were long awaited, finally got into the program. I'm confident that some time from now, may be next year or the followinig one, the interface will be faster.

Reply

• Oct 2, 2019 - 19:00

I know that, but anyway the uncompressed files are not that large. My largest mscx is about 17 Mb.The problem with big scores isn't saving (except when autosave gets in the way of some other process, which can be easily prevented by just increasing the autosave interval). The real problem is relayout, particularly in cases where it is not really necessary.

Reply

• Aug 17, 2019 - 15:44

I've found some things that take too long to get done and reported them and they were bugs that were fixed. This is in page view. In continuous view, you will inevitably see a slow down. There are certain things that force a system to be redone and since continuous view is one system, it will slow down. Continuous view is much better than it was in 3.0, where it was useless, but everyone admits it can get better. Someone needs to come up with a reasonable method for limiting what gets recalculated and/or drawn on each edit.

Reply

• Aug 17, 2019 - 17:06

To be clear, Recalculating it what is needed in continuous view is exactly what I did for 3.1 - as it applies to the measure-by-measure layout. So that's how it got so much faster at 3.1. But it's still the case that system-level elements like lines and I think so alignment of lyrics happen for the full score on each edit. In theory this can be improved as well, it's just more time and effort. I stopped when I reached what seemed to be a good knew in the curve (eg, 80% of the results with 20% of the effort).

Reply

• Aug 17, 2019 - 17:27

Marc, I hope you didn't take offense. Your efforts in improving continuous view are greatly appreciated. You made it usable and there were a lot of big bugs out there still that needed attention more than continuous view editing at that point. We've been involved in brainstorming sessions in the past perhaps at some point in the future someone will revisit continuous view and continue to improve it.

Reply

• Aug 17, 2019 - 19:25

Not to worry, no offense taken. I just find it useful to clarify things for future reference, starting point for further discussion, etc.

I still would like to see a good solution for staff spacing in continuous view, and I am disappointed I couldn't come up with a good totally automatic method that didn't pay too big a performance penalty. But I hope to return to that problem someday if no one else does first.

Reply

• Aug 18, 2019 - 13:05

Perhaps the only true complete solution to action triggering a full relayout that can't logically be avoided in continuous view is to implement threads.
User action 1 triggering full relayout would start the relayout in a thread, keeping the GUI thread idle and available to react to user action 2. If user action 2 triggers a full relayout again, that one will run and the first one will be cancelled.
That way the user interface would always stay responsive even for monster scores.
The layout thread must regularly update the screen, not only when work is finished.

Reply

• Aug 18, 2019 - 13:28

If I take you correctly, this is in line with my most recent thoughts on the matter. (I think)
I have been thinking that it "may" be possible to implement an "idle CPU" update of off-screen elements.

Thought no. 1 is that continuous view takes into account the entire score, yet only a small portion of the score is actually visible at any one time. It's not entirely necessary for edits to be applied to the entire score until such time as they are actually needed. Is it possible that off-screen edit details could be cached while CPU use is high?
Thought no. 2 is that when we are working, we are not "always" working. We spend a lot of time looking at our work and evaluating its progress, and also spend time referring to notes and reference sheets, etc. At such times CPU use is low, and MuseScore could take advantage of these pauses to implement cached updates to off-screen elements.

There may very well be reasons that this is entirely unworkable, but it is perhaps something to consider.

Reply

• Aug 18, 2019 - 15:28

As explained above, it's already the case (as of 3.1) that only the portion of the score that actually changes is recalculated in continuous view. So there is really little to be gained by moving calculations "offline". The wins left to be had are being smarted about what in fact has changed. Right now we still look at lines as needing full recalculation when really only their offsets (which are relative to the beginning of the score in continuous view) have changed in most case. Lines that span the changed measure may have changed length and thus need a fuller recalculation. Just improving that is far simpler and likely more effective than rewriting the layout algorithms use multiple threads or whatever.

If you mean, to handle the operations that do indeed affect the entire score, then indeed this approach could be helpful, but it would likely be an enormous undertaking to produce a relatively small win in real life terms (how many scores, how many operations are we talking about - and then scrolling would be much slower since things aren't precalculated).

Reply

• Aug 18, 2019 - 18:31

I am (was) indeed thinking of operations which may affect the entire score, but I take your point that the win is likely to be very small within the larger picture. One point, however, is that over the course of a session, successive edits may sometimes compound the effects of other operations, (In terms of necessitating a global re-write) while some may even cancel out such need. Holding some operations until score navigation requires redrawing the page, or until the CPU is not otherwise occupied, may have a small benefit towards enhancing the performance of MS using larger scores.

If I could be devil's advocate against my own suggestion, though, I foresee at least two potential difficulties with delayed implementation of edits/re-draws.
1. How is MS to know how much idle time to allow the user before implementing a cached edit? There is always a possibility that the re-write will take place at a time when the user is beginning a new edit, thereby negating the savings of CPU resources.
2. Having a number of unfinished operations open in memory could be a mine field for data corruption in the case of a system crash.

In terms of overall benefit, I don't think my suggestion would amount to very much on its own, but may be a part of a larger plan to improve performance of MS in the future.

Reply

• Aug 19, 2019 - 07:46

@toffle
You say "How is MS to know how much idle time to allow the user before implementing a cached edit"
?
How can that be a problem compared to the current situation?
Currently MuseScore always recomputes the necessary parts.
Always.
So why would we need more fine analysis and guessing work to launch that exact same recomputation in a different thread?
Actually we would do less recomputation because if meanwhile a new user action requires another relayout, the current one would be cancelled and only the new one done.

Reply

• Aug 19, 2019 - 07:41

@Marc "then scrolling would be much slower since things aren't precalculated"
That's not what I meant.
Same amount of calculations would be done at same speed as today, just that they would be in a distinct thread to avoid blocking the UI.
So if when you scroll the calculation is not finished yet, then indeed you wouldn't see the result yet, but that's true in the current way as well (and even worse as currently you can't scroll or do anything at all until calculation is finished).

Reply

• Oct 3, 2019 - 15:42

What I've thought about is calculating only what is needed is only a temporary fix to the speed issue. As you keep automatically compressing the adjacent measures to compensate for the increase measure size of the edited measure, they will eventually become so compressed they can't be compressed any farther. The program will not be aware of this during each edit. Therefore, the program will still check to see if those measures can be resized, then when it figures out it can't, it moves on to the next measures. This creates two problems:

  1. It wastes time recalculating for every single edit, the measures that were already compressed enough to compensate, rather than simply automatically moving those measures and calculating the ones that HAVEN'T been compressed; and
  2. Theoretically, eventually all the measures will be compressed to the point where virtually the entire score's worth of measures (or nearly all measures) will have to be calculated and re-positioned.

I believe, to counter the first problem, there should be a flag/Boolean variable that is returned true when that specific measure is compressed to its minimum size. Then, it can simply check to see if the variable is true or not. If it's not true, it is simply resized (then the remaining size that has to be moved leaks over to the next measure). If the measure is compressed to its maximum, the variable gets set to true. Then, if the computer checks that it IS true, it simply pushes that measure over and then moves on to the next measure. This eliminates the need of performing numerous comparisons to the size of the measures.

A function like this could be used:

    void move(measure, measureNumber, space) {
        measure = getMeasure(measureNumber);  // Get the measure itself as a variable

        moveMeasure(measureNumber, space); // Run the function(s) to physically move the measure (regardless of whether or not the measure can still be compressed, the current measure has to be moved.

        if(!measure.maxCompress) { // Check the boolean 'maxCompress' for the measure selected to see if it's not true to see if it can be compressed.

            int spaceLeft = compressMeasure(measureNumber, space); // The function to actually compress the measure could also return the space leftover that still needs to be allocated for the change.

            if(spaceLeft > 0) { // Check if there is still space to be allocated for the measure size
                move(measureNumber + 1, space); // Call THIS function for the next measure
            }
        }

        else {
            move(measureNumber + 1, space); // Call THIS function for the next measure
        }
    }

I also uploaded the sample code as a text file.

The second issue, I'm not sure how to fix. Perhaps there could be a refresh button that manually re-layouts the entire score (this is okay since it's intentional), resizing all the measures to a comfortable size. That way, some measures can be compressed again? This problem, I'm not sure about.

I hope I've been helpful. I appreciate all of your hard work to make Musescore better!

Attachment Size
codeSample.txt 983 bytes

Reply

• Oct 3, 2019 - 15:56

I'm not sure what you are referring to here What "compressing" of measures do you mean? There's no such thing unless you mean the "stretch" command, which has nothing to do with saving. If you mean the compression of the file, that's not done a measure at a time, that's just a standard ZIP-style compression function we call. It's not as far as I know a significant part of save time. As far as I know the reason save is slow on large scores is really just about the fact that we force a full layout (or two) as part of the operation to make sure the data is consistent and to workaround a few problems that otherwise occured. Chances are very good those problems can be addressed directly and the need for full layout on save could be removed.

But again, I don't see that compression of any kind is relevant here.

Reply

• Oct 5, 2019 - 18:35

I don't see what the physical width of a measure on a page has to do with the speed of working with large scores. I also don't see why you'd need to do anything special to make a measure narrower after editing another measure to make it wider - this happens completely automatically already, no additional work required on the part of the use,r no additional code required in MsueScore.

Sounds like you are talking about something pretty unrelated to what is being discussed here, if you would to explain more about what it is you are talking about with measure width, you might want to start a new thread to avoid confusion. And then attach a score and explain in more detail what you are talking about.

EDIT: Hmm, actually, maybe you are referring to continuous view and the need to update the layout of measures to the right of a modified measure? I already implemented this optimization a few months ago, we don't do the full layout but only add the offset. So that much is already done. Any further speed improvement would come from eliminating other calculations, like what we currently have to do for lines, beams, and other elements that need to be laid out system by system rather than measure by measure.

Reply

• Aug 19, 2019 - 01:53

Now that MuseScore is a highly refined application in terms of overall functionality (it is difficult to find anything that cannot be implemented even as a workaround), I find this topic of fundamental importance since it is the single major difficulty we, users, find when working on large scores (many parts and measures).
I think there are two different areas for potential improvement: 1) continuous view and 2) page view.

1) Continuous view is mainly used for inputting notes, symbols and text so it is closer to an internal structural representation than to a final rendering. In projects with more staves than those that fit on the screen, it allows seeing more music at a time since the page and margin separation do not interfere. Even if it is possible to make layout adjustments, which in recent versions seem to propagate without glitches to the page view, I think the imposibility to see the complete result (including margins, real size, system and page breaks, etc.), advises against using it for formatting. So layout changes affecting the overall score should be kept at a bare minimum.
I think continuous view shouldn't be considered a single oversized system, but a system on a page that has the width of the available horizontal space on the screen, or, at most +/- 1, additional screen to allow for smooth short range scrolls. As to staff spacing, I can think of two possible solutions:
a) To have a score-wise variable for each staff (or a vector with as many components as staves) to be updated each time a spacing change happens (for instance, adding a spacer, automatic spacing increase by autoplacement). The update consists in checking if the new required distance is larger than the last registered value. If true, update. This variable could be saved in the mscz file so that it doesn't need to be calculated each time the file is open. In case of a crash, it would have to be recalculated to prevent inconsistencies (with some extra delay, which is acceptable after a rare event as a crash). This variable could be used to compute the layout of the current visible area plus +/- one screen (i.e., 3 screens) after each scroll or whatever change. As the relayout affects 3 screens, it produces only unperceivable delay.
b) To have a staff distance property for the continuous view whose only impact would be for rendering the continuous view. This could be adjusted interactively from the contextual menu that opens right-clicking on a free part of a staff. It would be useful when the space between staves is overcrowded (for instance, several ledger lines, dynamics, hairpins).

2) Page view is used, in the case of many staves per system, to give the final formatting, moving misaligned hairpins, add spacers, system breaks, page breaks. One problem, either arising from memory issues or some inefficiency in the code, is that in many reported cases the program freezes during simple changes that potentially may require a complete relayout, but actually don't cause any relayout at all. For intance: adding a spacer never causes any change other that its insertion, since by default it is 3 sp high, which is less than the default distance between staves, 7 sp. I'm not sure that this relayout takes place, but the observed behavior is as if it did, while other changes which could potentially cause the need of massive relayout are much more responsive (for instance system break or page break). Only when the spacer height is changed it could cause relayout. If I'm correct, a simple test could be performed to see if Spacer_height > Stave_distance and only if true, relayout. Even if it is true, the total effect measured as the sum of all stave heights, stave distances, system distances on the page should be checked to see if it is larger than the vertical space allowed on the page, and only then perform a complete relayout, and only in the forward direction.

Reply

• Aug 24, 2019 - 07:11

Through the discussion I've come to know that many troublesome delays are caused by auto-save, so I suggest maybe we can implement an info box showing something like "The score is too big, close auto-save?" when loading big scores?

Reply

• Aug 24, 2019 - 13:39

That's possible, sure. BTW, it really isn't the save itself that takes the time, it's the full layout (or maybe two?) that accompanies the save that takes the time. The layout is performed to make sure we can create a valid thumbnail, to make sure we get the right info about multimeasure rests, and other things. It's probably also worth considering if there is a way around that.

Reply

• Aug 24, 2019 - 17:11

I would eliminate the thumbnail in backups and therefore any relayouts, they serve no purpose there except to slow down auto save. When the user saves the file for real MuseScore can then create the thumbnail. I realize this would mean rethinking a few things, but I would then turn my auto save back on, which I actually prefer.

Reply

• Aug 24, 2019 - 17:43

Unfortunately, it's not just the thumbnail that requires the full layout, it's also needed to get the full proper info for multimeasure rests. Maybe for hide empty staves too, I forget. And who knows what other hidden dependencies might exist. But in any case, I agree it's worth investigating further to see if there are opportunities for improvement here.

Reply

• Aug 24, 2019 - 17:45

Again, though, save is fast, its the layout that's the issue. Writing a whole new system to handle writing only the changes would almost certainly itself require a layout and would probably take months to work the bugs out of only to find it was still just as well. Almost no chance this would prove fruitful compared to the things I've identified.

Reply

• Aug 24, 2019 - 23:27

And I think maybe this is a general side effect of the using QT.

For example: Another (created with QT) software has difficulty to saving 300-400 MB soundfonts, while the entire user interface freezes and does not respond. Then suddenly it comes back to normal. But during this process, OS marks that software as "unresponsive" temporarily.

Reply

• Aug 25, 2019 - 03:31

MuseScore takes also lot of time, but really a lot (one minute or more), to load a soundfont as the HQ, which weights less than 100 Mb (the disk transfer may take 6 s). During this time MuseScore it freezes. This makes one think it has crashed. Only being patient enough to wait one can discover this is "normal". This may be a related issue. Another is the playback artifacts that causes a big score when trying to use this soundfont (many notes per unit time). This is a real-time problem, the MP3 export is correct (but takes several minutes).

Reply

• Aug 29, 2019 - 06:37

When I try to save my project, it undoes everything i've changed. this really sucks because sometimes i work for a few hours and save but reopen the score 10 seconds later and my progress is gone, im back to square one, saving the project did nothing

Reply

• Aug 29, 2019 - 13:45

Probably you saved to one folder but have another copy of the score in another folder. If you do a search of your computer for files ending in ".mscz", you should be able to find the version you saved. Also, when in doubt, before doing a "save", look at File / Score Properties to see what folder it will be saved to. use "Save As" instead if you wish to change this.

Reply

• Aug 29, 2019 - 15:49

Another possibility is trying to Save as, probably it will present the default folder where th files will be saved. With a bit of luck the other file will be there. Another is to look at File / Open recent. Hopefully your file will be in the list

Reply

• Sep 10, 2019 - 15:17

I see that adding time signatures also cause a full re-layout, but changing that to triggerLayout() doesn't make it any quicker... at least not obvious. Is there any other way we can speed up the layout process?

Reply

Do you still have an unanswered question? Please log in first to post your question.