Update on the Priority/Severity categories in the issue tracker
Hello fellow contributors,
Thanks to all of you helping to find bugs, fix them and check whether the fixes work! Thanks to everything you do we could make four patch releases with more than 170 issues solved.
We are looking forward to delivering a better product to the users. That's why we need a clear understanding of what is important to fix/do first of all.
Intro
Current list of P0 issues contains 80+ issues. All of them are "Critical" in terms of priority. It makes distinguishing between the ones which are really critical to fix including the meaning like "we cannot make a release without fixing the issues" impossible.
For example, speaking about the regression we had to fix in 3.0.4, the issue had P0-Critical Priority and S3-Major Severity. The same Priority and Severity we have for this issue. They are obviously not the same in terms of real importance.
Severity
I suggest revising all P0, P1, P2, S1, S2, S3 issues. The goal is to reduce the number of P0 issues to manageable 10-20 items to fit one page. We want to make the meaning clear and transparent to every user of the tracker both contributors and users.
| Category | Cases | Reproducibility, Frequency |
|---|---|---|
| S1-Blocker | Crashes and hangs; the issues which lead to data loss, damaging layout, corrupting scores; incorrect exporting data to external formats (PDF, MIDI, MP3, etc.); incorrect playback; inability to interact with the interface, broken interface | General cases affecting a lot of users; stable reproducibility scenarios with several different scores |
| S2-Critical | Same as above | Specific cases which were reported once, with one specific score, without stable scenario; the cases which affect a specific group of users |
| Regressions excepting the ones mentioned above; inability to translate some parts of the interface; interface bugs (incorrect wording, incorrect window sizes, etc.); inconsistent layout; not working built-in plugins; performance issues; layout drawing issues; changes which are not saved | General cases affecting a lot of users; stable reproducibility scenarios with several different scores | |
| S3-Major | Same as above | Specific cases which were reported once, with one specific score, without stable scenario; the cases which affect a specific group of users |
| Specific functionality issues (mixer, soundfonts, timeline, particular properties, navigation, etc.); playback inconsistencies (pauses, arpeggios, etc.) | General cases affecting a lot of users; stable reproducibility scenarios with several different scores | |
| S4-Minor | All other cases | Any |
Examples
[MacOS Mojave 10.14.2] Export Pdf with Muse Jazz Text all garbled and unreadable
The issue describes a specific category of users. The issue relates to "incorrect exporting data to external formats". So, set S2-Critical severity.
Crash when changing Time Signature in parts
The issue describes the crash which affects a lot of users with different use cases. So, set S1-Blocker severity.
Leading space adjustment for generated elements lost on save/reload
The issue describes the changes which are not kept in the score and don't survive save/reload. So, set S2-Critical severity.
Priority
Priorities point contributors (developers) to the issues which should be fixed first. 80+ P0 issues make focusing on the important ones impossible. Priorities are defined by core contributors who are interested in the constant improvement of the editor.
Suggestions
- All S1 issues should be marked with P0 priority
- If the number of S1 issues is not enough, add some S2 issues which relate to "General" cases to the P0 list
- Mark all S1 "General" case issues as P1
- Mark all "General" case S5-Suggestion issues as P1
Tags
Tags facility allows marking particular issues and classifies them by functionality, a particular part of the functionality, etc. The issues which are supposed to be easy-to-implement are marked with "easy" tag.
There are some examples of the tags: "mmrests", "arpeggios", "timesigs", "clefs", "accessibility", "cross-staff notation", "tuplets", "mixer", "play panel", etc.
Tags help contributors to choose the issues by the functionality they are interested in.
Conclusion
Should you have any questions or/and suggestions don't hesitate to express your opinion in the comments.
A lot of discussions happen in the public Telegram channel: https://t.me/musescoreeditorchat. Join to participate in hot discussions and have fun :)
Comments
In general, I definitely approve of the goals. I very much like the idea f having P0 represent something like, "the manageable set of highest priority bugs to fix for next release". Specifics about how we get there I figure will continue to evolve over time until we figure out what works.
That said, I suspect we'll find the definition of S1-Blocker as stated is too broad. People will see "crash = blocker" and not really catch on to the significance of the reproducibility & frequency. Also, defining "incorrect playback" and some others is going to invite S1 issues like "the downbow marking doesn't affect playback", which is of course wholly different from "all my instruments sound like pianos". Not a huge deal, we can always reassign severity when assigning priority.
One way to prevent "severity inflation" might be to replace subjective severity levels with objective categories:
Priority would remain something decided by core developers based on numerous factors including severity, number of users affected, and ease of implementation.
Aside: I think the "type" field has two many options and shares too much functionality with tags.