CI and mtests
Ciao.
Recently, "mscore/workspace" and "mscore/palette" mtests were disabled during CI test builds because no GLX could be initialized in the headless machines used by github actions.
I think I found a solution by using QT_QPA_PLATFORM=minimal:enable_fonts (see for example https://doc.qt.io/qt-5/qpa.html#qpa-minimal-plugins and https://www.yellowsubroutine.com/qt/qt-qpa-platform) here: https://github.com/musescore/MuseScore/blob/master/build/ci/linux/runmt…
However, in this case GLX is not required, but there is a crash due to the built-in freetype library (thirdparty/freetype). You can see that crash here:
https://github.com/AntonioBL/MuseScore/runs/1265244271?check_suite_focu…
Compilation and run of "workspace" and "palette" mtests work when using USE_SYSTEM_FREETYPE="ON" with make instruction, as seen here:
https://github.com/AntonioBL/MuseScore/actions/runs/315549119
I also tried to update freetype from 2.6.1 to 2.10.3 and those mtests worked, see:
https://github.com/AntonioBL/MuseScore/actions/runs/317537706
(I didn't check if compilation still works for Mac and Windows in this case).
In summary, if we want those tests to be performed, we need to use QT_QPA_PLATFORM=minimal:enable_fonts (or "offscreen" platform) and system freetype or an updated bundled freetype.
What do you think?
I don't know if mtests are still considered useful. They helped a lot in the past in finding errors and regressions at PR time. They were also compiled with address sanitizer turned on to catch possible memory errors, but now they are compiled in release mode. And gtests were added.
So, as an additional question, what's the future of mtests?
Ciao,
ABL
Comments
For the 3.x branch, the mtests will work as works.
So, it would be great if you do PR with your solution for 3.x branch.
Though, we planned releases only 3.6 and all.
For master, exactly this is 2 tests not actual, because they tested code from
mscore, which will be deleted coming soon.In general, for master, mtest will work as works for testing
libmscore.> They were also compiled with address sanitizer turned on to catch possible memory errors, but now they are compiled in release mode
Good point.
At the moment test run combined with build to get pull request artifacts.
I think we can build in debug mode for pull requests.
In reply to For the 3.x branch, the… by igor.korsukov
> So, it would be great if you do PR with your solution for 3.x branch.
Ok, but which one?
I think it is maybe better not to update the bundled freetype in 3.x, in order to avoid possible regressions. Or do you think it is worth updating this "internal" library (at 2.6.1 while upstream is at 2.10.3)?
For the other possible solution, tests are now run on the same job which creates the binary distribution, so
USE_SYSTEM_FREETYPE="ON"would also affect the binary executables.Can I split the mtests into a separate job?
By the way, when activating addresssanitizer or when running under valgrind all tests pass without crash, so there is maybe a thread race problem, or something similar, for the bundled freetype (but thread tools such as helgrind do not show anything).
> In general, for master, mtest will work as works for testing libmscore.
So, if I understand correctly, mtests will remain, but they will not test the GUI-related part. Will it be tested with googletests? Or will the workspace and palette mtests be updated after all the changes will be implemented in master branch?
In reply to > So, it would be great if… by ABL
> Can I split the mtests into a separate job?
I think, better make a separate job for mtest and set
USE_SYSTEM_FREETYPE="ON"I wanted to avoid it, but I think is better, than update freetype for 3.x, plus this allows switch on addresssanitizer and etc.
> So, if I understand correctly, mtests will remain, but they will not test the GUI-related part. Will it be tested with googletests? Or will the workspace and palette mtests be updated after all the changes will be implemented in master branch?
The current mtests will remain while it being helpful.
We not expecting new mtests, but do not prohibit.
In-house team will be using googletest.
Palette and workspace tests will be rewritten on googletest
Anyway, this is a great job! This may be useful in the future.