Saving segment positions with MuseScore 3 leads to elements with sx attributes being mostly 0

Hi Marc,

with that functionality, we can generate .png images from a MuseScore file + have a corresponding mapping of wall-clock time to position (note or measure rectangle) on the image. We need this for a (still beta version of) our score follower. You might remember Thomas and Nicolas doing demo's with this technology connected to MuseScore several years ago.

We actually also have C++ code to import scores from a user's MuseScore account (after authorizing with OAuth), but apparently the MuseScore API is now no longer working, so this now also no longer works in our application. It was convenient for a user to import his/her own scores that way, but I can understand that decision...

So, the code was changed to start MuseScore locally in a separate process to perform the conversion and generate the .png, .wav, and .mpos/.spos files locally on the user's own computer from a local MuseScore file. And then I noticed the .mpos file is still OK in MS3, but the .spos file has these widths of 0 (which was not the case in MS2). It looks like the bounding box calculations for notes doesn't work any longer as it used to and leads to zero rectangle widths for the elements (except for rests, I think)?

I had a quick look at the code in MuseScore, and noticed some things are done while iterating over the score elements, but it wasn't clear to me what was going on exactly and why this would now fail in MS3...

Hopefully there is a simple fix?
In any case, thanks for your fast reply.

Kind regards,
Koen

Hi mirabilos,

I believe everything we talked about is already in my original post + my comment above from June 29, 2020 with a bit more use case info.

I also attached the files to my original post + the commands to reproduce the issue showing the difference between MS2 and MS3.

Maybe this phrase I sent to Emmanuel as part of our private conversation helps for you to understand the use case better (but to me, it's essentially the same as above, just in other wordings):

"What our music follower needs is a list of events with a timestamp and a rectangle I can draw over the score image. The rectangle should be an indication of what is currently playing, so it could be a single note or a (partial) chord, ... I would expect that that rectangle encloses the note or the chord.

I think the way it was when I was still using MS2, was that the width of the rectangle was more or less the width of the note or the chord, and the height was fitting the system height, so you would get a rectangle that was always the same height (it doesn't matter if it's a low note or a high note), but the width would depend on what was in the rectangle (note, chord, ...) We also extend the height a little bit so it exceeds the highest and lowest horizontal staff line."

I can also show you the relevant piece of our code that is consuming the .mpos or .spos files (I had already shared the relevant developer note in the other post at https://musescore.org/en/node/311914 but maybe this helps too):

// Add the (place) regions
const double MuseScoreDPI = 360;
const double ExportDPI = m_ImageResolution;
const double xFactor = 1.0 / (pageWidth  * (12 * MuseScoreDPI / ExportDPI));
const double yFactor = 1.0 / (pageHeight * (12 * MuseScoreDPI / ExportDPI));
const double extraFractionAtTopAndBottom = 0.2;
for (XmlElement* elementElement = elementsElement->getChildByName("element"); elementElement != nullptr; elementElement = elementElement->getNextElementWithTagName("element"))
{
    MusicFollowingPlaceMap::Region region;
    region.ID = elementElement->getIntAttribute("id");
    region.Page = elementElement->getIntAttribute("page");
    region.Type = "Rect";

    /* More info on the coordinate conversions for the positions of the elements / measures:
    //     mpos.x = frac.x * PageWidth  * (12 * MuseScoreDPI / ExportDPI)
    //     mpos.y = frac.y * PageHeight * (12 * MuseScoreDPI / ExportDPI)
    // or also:
    //     frac.x = mpos.x / (PageWidth  * (12 * MuseScoreDPI / ExportDPI))
    //     frac.y = mpos.y / (PageHeight * (12 * MuseScoreDPI / ExportDPI))
    // where:
    //     mpos.x / .y are the x and y coordinates in the .mpos file extracted from MuseScore
    //     PageWidth and PageHeight are in pixels
    //     MuseScoreDPI is 300
    //     ExportDPI is 100 by default      */
    double x = elementElement->getDoubleAttribute("x");
    double y = elementElement->getDoubleAttribute("y");
    double sx = elementElement->getDoubleAttribute("sx");
    double sy = elementElement->getDoubleAttribute("sy");
    float X = (float)(x * xFactor);
    float Y = (float)(y * yFactor);
    float W = (float)(sx * xFactor);
    float H = (float)(sy * yFactor);
    region.Rect.Set(X, Y - H * float(extraFractionAtTopAndBottom), W, H * float(1 + 2 * extraFractionAtTopAndBottom));
    display.PlaceMap.Regions.push_back(region);
}

Note as already mentioned in the other post as well:

"But I also see in the code surrounding it that we currently set a const as follows:
const double MuseScoreDPI = 360;
and I found a commit log message (from last June actually) saying this:
"adjusted assumed MuseScore export dpi for PNG files to 360 (default value for MS3; was 300 before)
--> fixes issue with incorrectly placed bounding boxes""

For me, the mpos file still works fine (even though there is that strange factor 12 in there) as we use fractional values anyway. So for me, there is no real need to change the .mpos file (unles you want to make it better/more general for some reason).

This post was about the .spos files which are broken in MS3 and were still working fine in MS2.
I think I gave all the info I can give...

If I understood it correctly from what I read, Emmanuel already did a lot of work to get a fix for this, but due to some disagreements and then a commit of untested code you had made, things are now a bit in a "twilight zone", right?
I don't know the exact details (and I should not interfere in that) but I just hope my input here helps to get to a fix at some point, hopefully with not too many changes.