QML Palette: Design and Accessibility

Congratulations to the team on the new palette design soon to be released in MuseScore 3.3! The previous palette design was written in Qt's traditional C++ framework, which is still used for most of MuseScore's code. The new palettes are the first major feature to be written in Qt's JavaScript-based QML language, which until this point has only been used for MuseScore's plugins. The switch to QML should bring many benefits over time, including increased flexibility in design, and better scaling across devices and platforms.

The palette refresh already includes new design elements that will bring improvements to all users. Thanks especially to the team for taking time to ensure that the new palette is keyboard-navigable, and works reasonably well with a screen reader for the benefit of users who are blind or partially sighted!

However, there are still improvements that can be made, both to accessibility for keyboard users, and also to the visual design for sighted mouse users. I shall give a summary the new design followed by my own points and suggestions.

Current Design

Model

Like the previous C++ palettes, the recent QML palettes offer a "tree" of musical symbols. The branches of the tree are the different categories of palette (Clefs, Key Signatures, Dynamics, etc.). The leaves of the tree are the symbols (or "palette elements") that fall under each category (e.g. D Major is in the Key Signatures palette). The palettes are initally collapsed, but can be expanded to reveal the symbols they contain.

View

For sighted users, the symbols within each palette are displayed as a grid, but this is purely a visual trick to fit more elements on the screen and reduce the need to scroll. There is no special significance to the different rows and columns of the grid, which can be seen by the way the layout changes when the palette's width is changed (as many elements are drawn on each row as will fit). This is important because it affects accessibility for people who cannot see the grid.

Old "C++" Palette	New "QML" Palette

Search

As with the old palettes, a searchbox exists to quickly filter palette elements by name. However, the position of the searchbox has changed: it is now placed above the palettes tree rather than below. This makes sense because you want to search before you access the tree, not the other way around. Placing it above has also removed the awkward gap that used to exist between the palettes and the searchbox when the palettes were collapsed.

Add Palettes button

The Basic/Advanced workspace switcher has been moved to the main toolbar, which is sensible because it affects more than just the palettes. Instead, we now have an "Add Palettes" button that appears to the left of the searchbox. Pressing this button reveals a popup widget that lists all palettes that are hidden in the current workspace. The user can choose to re-enable some or all of the hidden palettes - an ability that was missing from previous versions of MuseScore - or to create a new one from scratch via a "Create custom palette" button at the bottom of the widget.

More elements button

Instead of the final "???" mystery item ("Show More") that was present in some of the old C++ palettes, the bottom-right grid cell in each palette is now occupied by a "More" button. Rather than opening the full Master Palette, pressing this button reveals a popup widget that lists only the symbols that have been hidden from the current palette. As before, the user can add them to the palette or directly to the score.

Additional options are displayed in this widget for certain types of palette:

• Key Signatures palette - there is the option to create a new key signature. This opens a separate dialog where users can drag accidentals to the desired position on a dummy staff.
• Time Signatures palette - there is the option to create a new time signature. This opens a separate dialog where users can enter a custom numerator, denominator, and beam grouping.
• Custom palettes - the user can add elements from any of the other (non-custom) palettes. This opens up the possibility of mixed palettes that contain symbols from multiple other palettes.

Context menus

Similarly to the C++ palettes, right-clicking on any palette reveals a context menu with options to save, load, customise or hide that palette, or to insert a new palette before it. Users are not always aware of the presence of right-click menus, and such menus are difficult to access via a keyboard or touchscreen device. The QML palettes solve these problems by making the menu available via a three-dots button to the right of each palette's name.

A similar menu exists for palette elements, though it is only available by right-clicking. However, the menu's "Delete" option is available via a trashcan icon that appears next to the palette's three-dots button when an element is selected.

Keyboard navigation

Accessibility in the C++ palette was handled exclusively through the searchbox. The user would type a few letters of a palette or symbol name and then use the up and down arrowkeys to navigate the search results. The actual palette tree was not accessible at all, nor were the customisation features. Happily this situation has now been almost entirely rectified, and almost every part of the QML palette is accessible to keyboard users by way of the Tab key.

Unfortunately the reverse problem now exists for keyboard users. Now that practically all objects are tabbable, including all palettes, palette elements, expand/collapse arrows, three-dots buttons, and so on, navigating the entire palettes tree can easily involve pressing the Tab key 50 times or more. The search feature can help to reduce this number considerably, but the situation is still far from ideal.

Appearance

The overall appearance is very similar to before, though it feels more polished in the details. QML has enabled the use of animations, so palettes expand and collapse in a fluid motion rather than an instantaneous jump. Palettes and palette elements can now be rearranged via drag-and-drop, and the remaining elements shuffle around during the drag operation to make room and give a live preview of the final result before you release the mouse button.

On a more practical note, palette names are spaced further apart than before, which makes it more difficult to click on the wrong one by accident - especially important on touch-based devices - though it does mean that fewer palettes will fit on the screen at once.

My Suggestions

1a) Move the Add Palettes button below the palettes

Placing the Add Palettes button in the top left corner gives it undue prominence. The button provides important functionality, but it is not needed as often as the palettes or the search feature. Logically, you need to look at the existing palettes before you can decide whether you need to add another, so it makes sense to put the Add Palettes button below the other palettes.

The Add Palettes button could be anchored to the foot of the dock area like the searchbox used to be, or it could take the place of a final palette, like a "More Palettes" palette.

1b) Make the Add Palettes button a More / Fewer toggle

Let's assume the Add Palettes button is replaced with a "More Palettes" palette as outlined in (1a). Pressing it could have the effect of temporarily adding all the hidden palettes beneath it. The "More Palettes" text would change to say "Fewer Palettes", and pressing it again would hide the extra palettes again. This does away with the need for a separate popup widget to add hidden palettes.

While visible, the extra palettes would be fully functional. The user could expand them to access the symbols they contain, or drag them above the "More Palettes" / "Fewer Palettes" divider to make them permanently visible.

2) Make the "More" button a More / Fewer toggle

Similarly to above, the "more symbols" popup is not strictly necessary because the extra symbols could be displayed in the palette below the "More" button. The "More" button's text would change to "Fewer", and pressing it again would have the effect of hiding the symbols below it.

While visible, the extra symbols would be fully functional. The user could add them directly to the score, or drag them above the "More" / "Fewer" divider to make them permanently visible.

3) The three-dots buttons are visually distracting

A simple solution would be to use a lighter shade of grey for the "..." to make it less distracting. Alternatively, the three-dots button could be hidden until the user selects a palette, expands it, or hovers the cursor over it.

4) The elements context menu needs a button

A three-dots button could appear in the top right corner of elements when they are selected. It could also appear when the cursor is hovered over an element, but care must be taken to avoid accidental presses. Alternatively, the menu button could appear elsewhere in the palette, like the trashcan icon does now.

5a) The entire palette tree should be one tabbable object

Issue: #296154: Palette tree should be a single tabbable object

Navigation within the tree should be done solely with the arrow keys. The user should be able to return to the searchbox at any time simply by pressing Shift+Tab once. Pressing Tab again should put the user back in the tree on the same item as they were on before they pressed Shift+Tab.

5b) Up & Down keys to change item, Right & Left to expand/collapse

Currently the user can navigate up and down the palette categories with the Up and Down arrow keys, expand them with the Right arrow key, and collapse them again with the Left arrow key. This works already, and it is excellent!

However, once a palette has been expanded, the user must switch to using the Left and Right arrow keys to navigate the elements inside it. It would be much more familiar to blind users if the Up and Down keys were used exclusively for navigating, and Left and Right used exclusively for expanding and collapsing.

Remember, blind users cannot see the fact that the elements are arranged in a grid. To a blind person, the palettes will appear as a tree view. If navigation of tree views is unfamiliar to you then I suggest you play around with the template chooser in the New Score Wizard, as that is a classic example of a tree view.

An additional benefit of doing it this way is that the Left arrow key can be used to quickly exit a palette in much the same way that Shift+Tab should be used to quickly exit the tree (see 5a).

5c) Accessing buttons within the palette

As a side-effect of (5a) and (5b), the trashcan and three-dots buttons will no longer be reachable by the keyboard. Convention wisdom is to put these buttons outside the tree so that the user can tab to them. This means that the buttons would be shared by all the palettes, rather than each palette having its own as is currently the case. The buttons would only affect the selected palette or palette element.

An example of this occurs in the Instruments dialog. First choose your instrument from the tree, then press the "Add" button (located outside the tree) to add that instrument to the score. Notice how there is only one Add button in that dialog, not one for each instrument.

Of course, moving the trashcan and tree-dots buttons outside the tree would be a shame for sighted users. Fortunately it is possible to get the best of both worlds by keeping the buttons in the tree, but disabling tab access to all of them except the ones that belong to the selected tree item. This could go hand-in-hand with (3).

6) Reorder palettes and symbols with the keyboard

We need a way of reordering items within palettes that does not rely on drag-and-drop. Short moves can be facilitated via "Move up" / "Move down" options in the relevant context menus, but it would be tedious to move an item a long distance by this method. A better method would be to select the item to be moved and press a "commence move operation" shortcut, then select the destination and press a "confirm move operation" shortcut.

7a) Close on Enter

When the user adds an element to the score, focus is given to the score, yet the palette remains open. This is fine for mouse users, but for keyboard users it means the F9 shortcut must be pressed twice to return to the palette. The solution is to close the palette when an element is added to the score via the keyboard, but to leave it open when an element is added with the mouse.

7b) Retain focus object

Currently the F9 shortcut gives focus to the searchbox. This is good the first time the palette is opened, but on future occasions focus should be returned to whichever control had focus when the palette was last closed. This also applies within the palette tree: the selected item should not change on closing and opening the palette dock widget.