Musecore/scorefile Score::saveFile() -- chance to use multiple autosave files?
Hi,
I was reading this code above and wondering, maybe it would be good to maintain multiple backups (autosave files) while a score is being edited in the session? I mean, appending a random sequence of numbers to each autosaved filename - then, of course, dropping all of them, after saveFile() exits successfully - and the written score file is closed ok in MuseScore.
something like "temp_score-XXXXXXX,mscz (or ,msczx) , being XXXXXXX a simple random number.
This way the users would be able to recover at least a previous autosaved file before a crash, if the crash was in a middle of a autosaved file write.
I think that his would just involve updating the temporary "session" file describing the opened scores, with the new filename , on the autosave timer callback (so keeping track of the last written autosave file for the score)
Comments
In theory, all saves already go to a temp file and then get renamed only at the end after success. However, the number of reports of people ending up with empty files is distressing - something still isn't working right despite this. So far no one seems to have any clue how this happening. But it would be great if someone could figure it out.
In reply to In theory, all saves already… by Marc Sabatella
Marc, one question. At any given time in a score editing session, how many files are actually opened at the operating system level?
I count these: the original score file, the backup file (that ones that has a "," in the extension), and the one only tempfile created by the autosave ((*) but this last one, only when the autosave kicks in). Am I wrong?
My proposal would work as follows:
-User opens a score - the program creates a "session" file , with the "dirty" flag = 0 (no changes). No tempfile yet.
-On the 1st autosave "kick-in" - If there are changes in the score, the program creates and writes the autosave tempfile and closes it - it currently do this well (*).
-On the 2nd autosaves - if there are changes, the program creates another tempfile, writes, and closes it.
-On the 3rd autosave - if there are changes, the program creates another tempfile, writes and closes it.
If a crash happens exactly when the autosave was writing into the current tempfile (or when it's renaming the backup file), at least you can have the above previous tempfiles on disk, that, in principle, should be consistent, as they were properly closed.
This would not cover the case if a crash happens before the 1st tempfile writing, but at least minimizes the chance of losing all the work done in a long editing session.
(*) ps: edit: I'll correct myself :-) . At any given time, while not being used, the autosave tempfile, currently, is properly closed: I had a score opened, waited, did some changes, MuseScore created and written over the tempfile the score. Then, I opened a command line, and deleted the tempfile - it deleted it, so it was certainly not opened (obviously it would we opened again at a point, but MuseScore detects its absence, and opens a new file instead). So it appears there is a small window of the autosave kick-in that opens the chance of a crash to make its harm to this file. Hence the proposal, because currently only one autosave tempfile is created (until the user explicitely saves the score, of course).
In reply to Marc, one question. At any… by mdi1972
The backup file is never actually opened as far as I know - it's just your original score renamed when you first re-save it during a session.
Each save opens a temp file, writes it, then renames it to the original name.
Not sure what problem you are trying to solve here. Is your concern solely about what happens if a crash happens during those few millseconds while the autosave is being written? Do you have reason to believe this has ever actually happened to you? Note in such cases your actual score should still be fine, o you've only lost the work since the last save, not all work in the session.
In reply to The backup file is never… by Marc Sabatella
I've never suffered a corruption in 4 years, that's not the reason I've posted this issue here.
The reason I've posted this is because, as you said, there are a number of posts of people reporting 0-bytes, or irreparable scores, as you well know. And, seeing the code (and even after reading your last reply above) I reinforce the idea that (as an added safety measure) it would be good to maintan more than one tempfile, because maybe not all people do an explicit "Save" at intervals (re-save) during a long editing session.
I think perhaps some of those misterious corruptions ("file is beyond repair..") reported in the threads may be explained by the fact that a crash could happen in those few miliseconds, or while the tempfile is renamed . So that's the reason I think that using more than one tempfile would be beneficial, as it will increase the chance to not lose all the work in the session, as currently MuseScore only uses one tempfile.
I'm not sure that for some large scores the write and rename last only a few miliseconds. First, it has to be zipped, and recall there may be antivirus or antimalware programs in place in the environment of the user which in the background observes this activity in the tempfile and delays the whole process a bit, to put an example.
In reply to I've never suffered a… by mdi1972
I've just opened #271185: Scores corrupted beyond repair, containing nothing but zeros to collect information about that problem, like steps to reproduce
In reply to I've just opened #271185:… by Jojo-Schmitz
buf. Steps to reproduce you said... well, I'll do my best on it.
In reply to I've never suffered a… by mdi1972
Not the zero-filled are not the auto-save files. They're the real files. And if the crash happens while writing the temp file, you are completely safe - your original file is still completely intact. And renaming a file is an autonomous operation - it won't result in a zero-filled file. So this cannot be the cause of the zero-filled files. And in the case of the zero-filled files, the auto-save file would generally still be perfectly fine, if it still existed. The issue here, when it occurs, isn't that the auto-save file is bad, it is that it is gone, because MuseScore normally cleans them on successful close.
So what you're proposing wouldn't help that I can see. It wouldn't avoid the creation of these zero-filled files, nor would it make it easier to recover afterwards using the auo-save. In order for the auto-save to be more useful, we don't need more of them per score, we would just need to actually keep one. Which could take up an awful lot of room over time, but perhaps some automatic cleanup could be implemented.
In reply to Not the zero-filled are not… by Marc Sabatella
Oh. Well, sorry - as I didn't experienced any crash myself, I didn't know that it's not the tempfiles which are zero filled, the ones that are being reported actually. My entire proposal was based in a wrong assumption.
Disregard it.
In reply to Oh. Well, sorry - as I didn… by mdi1972
I think you might be misunderstanding. The tempfile almost certainly was zero-filled - I'm assuming the problem was in writing the tempfile, not in renaming it. But the point is, the autosave is not generally zero-filled - it simply doesn't exist, because by the time the user realizes he has a zero-filled file, the auto-save was long since deleted since no actual crash happened. If a crash had happened, then most likely either the original file or the auto-save or both would actually be fine.
Don't get me wrong - I definitely applaud the initiative in trying to find a solution to this vexing problem. I just am not saying how creating more autosave files actually would help, since the problem is seldom a bad autosave.
In reply to I think you might be… by Marc Sabatella
Yes, Marc, I don't get you wrong, I perfectly understood your correction. Thanks for claryfing, really. I'll do my tests, after all, I'm interested to get (by myself) a 0-filled score...;-)