MuseScore ScreenReader Accessibility
StartOfMessage:
[This is my first blast through MuseScore 3.4.2 just to see what navigation keys work and get the 50,000 blue sky overview of what it looks like.]
Click MuseScore desktop Icon:
Heard the executable file path and Start center dialog followed by Online Community Mastering MuseScore complete online video extension for muse score 2.3? - new video course
[i am not recording this speech in detail but while alt tabbing between MuseScore and Notepad I am taking notes on what I hear in general related to navigation.]
First: the inverted "t" arrow keys on my U.S. English Standard Desktop Keyboard using NVDA - current version read nothing.
Next, the alt key does not bring up a Context Menu or any other menu.
Top of page, bottom of page and other navigation keys seem to do nothing either.
Hitting the tab key first read "Split Pane"
Hitting it again a few times after alt tabbing back into MuseScore I heard
open a score... Enter button
Close enter button
pane online community link
find sheet music... edit
search button
moonlighting followed by some characters and the name tim Hamelton or something to that effect:
Hitting tab again after bouncing back and forth between typing this Notepad document and MuseScore [using alt tab] I just was cycled between the same controls
but diferent what sounded like either music titles or forum titles like moonlighting..., silent night... etc:
[I hit alt f4 to close MuseScore but...]
A "Untitled Window" came up with Split button being repeated 4 times followed by Something about untitled treble cleff window and piano
[It appears to be a Edit Window for working on a score]
again arrow keys do not work.
I hit tab and heard
Inspector window visible checkbox checked
Automatic placement checkbox checked
other controls, spin buttons etc...
I Tabbed down to some unlabeled checkboxes - note someplace along the line the arrow keys started working. and heard
the Unlabeled check boxes followed by a flip direction button
[arrowing around continued to read various items and, or seemingly change values within certain controls without intending to.
Arrowing and tabbing also found other navigation features like create new score, play stop etc...
Everything jumbled together on one window.
I hit alt f4 to close MuseScore and a message asking me if i wanted to save the project came up.
I did not click anything in this message but Alt Tabbed back to type in this document...
NVDA typeing became quite slow perhaps indicating some unintended connection between museScore and NVDA - dont know.
I attempted to Alt Tab back into MuseScore but it was nowhere to be found, not even in Task Manager as far as I can tell (at least the message, dialog or whatever was not there under MuseScore process) but I had not closed the message asking if i wanted to save the project yet.
I then Restarted NVDA and the typing speed improved again.
Summary:
I believe the older version of MuseScore had a fairly clean initial user interface and a standard functioning Menu Bar with operations and categories for what features MuseScore offered - at least that is what i remember in general.
This version does not seem as accessible [or, at least, screen reader user friendley]
the current UI seems confusing and does not adhere to accepted and established navigation controls standards.
The current situation may be due to either unhandled or unreported accessibility tree properties and methods of the current user interface objects.
I have heard from a friend in education that one of the third party libraries used by MuseScore may not be very screen reader accessible.
note, I believe NVDA may also use this third party package.
So I am also wondering about potential conflicts concerning the accessibility trees especially considering cross process or background handling of diferent dll methods and objects with the same names or identifiers and how calculations and reporting of methods and object properties and events are handled by various operating systems regarding this open source package.
Anyway, this is my first 50,000 foot review of the accessibility and screen reader usage of NVDA with MuseScore.
I imagine there are work arounds to get through this user interface using a screen reader to get the job done but in my opinion it is not ready for prime time and, unless the third party open source folks creating the objects adhere to web and general user interface standards (perhaps w3c based) when modifying their software, it may never be since a screen reader has to rely mainly on what is reported in the accessibility trees.
I had a thought, for at least 1 second, smile, about converting MuseScore into a project using standard windows operations and controls via visual studio and vb.net but...
MuseScore appears to be a very complex and large project with several .net unmanaged third party libraries and over 1,000 modules of the same.
I would imagine converting it to a more accessible platform and making full use of UIA or IA2 accessibility trees and features (enforced as possible upon any third party vendors) would be most time consuming, dificult and perhaps not even doable by reasonable standards but I dont know.
All this said, does it look like you will be able to get back to a more w3c ish standard user interface and something like a standard Menu Bar using the current third party technicals?
Has anyone considered translating the project into a Microsoft Managed Environment using something like C++, the standard managed Windows or Web controls such as in vb.net or C# via Visual Studio?
Microsoft is accessibility orientated and usually their controls and windows are screen reader accessible all be it perhaps not cross platform unless using one of their cross platform languages.
That said, It looks like it would be a huge undertaking with many complicated technicals and required custom windows or pages but may be doable not at all sure about that though
Finally:
Could you pros jus comment on my above musings - grin pun intended.
Your experience is the best sounding board out there.
I am just trying to pick your well experienced brains regarding potentials - what has been considered and what may, or not, be possible going forward if you were thinking of a 3 to 5 year development path for MuseScore and features and accessibility.
Rick USA
EndOfMessage:
Comments
Thanks for your comments! With the recent advances we have made in accessibility, more and more blind users are using MuseScore, so we're getting lots of feedback that should help us make further improvements still.
Did you read the section on accessibility from the Handbook? It explains in pretty great detail how to find your way around the interface. See https://musescore.org/en/handbook/3/accessibility.
Two important things you'll learn from there relevant to your comments here:
1) the Start Center is accessible but awkward to navigate indeed, you are probably better of just closing it (Esc does the trick)
2) The navigation commands you will mostly want are Alt+Left and Alt+Right. These move through the score one symbol at a time, reading virtually everything. When you tried Left and Right, it only moves through notes and rests, and they only work if you already have a note or rest selected, which you don't by default. So you need to use Alt+Right to get started on the navigation. It will read the title of the score, the treble clef and time signature of the first staff, then finally the first rest (since the score is empty, it has nothing but rests).
Also, Tab will take you on a pretty lengthy tour of the main window, going through the Inspector, toolbars, etc. We definitely want to make this better organized. But - none of this has changed in any way from any older version. And the Start Center has been around for many years as well. So really, nothing should have changed regarding this initial experience. Also, you can at any time return focus to the score window by pressing Esc.
More importantly, though, we've actually designed the interface so you don't normally need any of the things you are finding when you Tab around. Virtually everything worth doing is also accessible from the menu or via a keyboard shortcut. The main exception is the palettes, which were not accessible at all in older versions but as of 3.3 they are. Meaning, for the first time you can add dynamics and other symbols to your score - this was not possible until a couple of months ago. So do be sure to read the section on Palettes specifically in the accessibility documentation to learn how to take advantage of this.
BTW, the dialog when you tried to quit is probably from some setting getting changed when you were tabbing through the Inspector, although that's really just a guess. Not sure what happened that allowed MsueScore to clsoe without you deliberately answering in that dialog, but that's probably a fluke, normally that works perfectly. Could be you accidentally pressed Esc in there somewhere. Unless you can reproduce that behavior reliably, I wouldn't assume there is an actual problem there.
Regarding the implementation, we use the Qt libraries for managing the user interface, and it is primarily based on MSAA but also includes some UIA, IA2 and AT-SPI features. This enables us to work cross-platform, and we have Orca working on Linux and partial support for VoiceOver on macOS. Your friend probably heard about very old versions of Qt that were indeed not very accessible, but current versions are pretty good - best cross-platform solution by far that I know of.
Anyhow, you are correct that MuseScore is a big and complex program with much to learn about it, but I think if you read through that documentation and try my suggestions here, you will start to get the hang of it.
In reply to Thanks for your comments! … by Marc Sabatella
Hi Mark:
I opened MuseScore, closed the Start Center Dialog window using escape and the Edit Window was there.
Hitting the alt key opened the menu bar as expected with file, view and all the menus ( Last time it did not work but I had been alt tabbing and floundering extensively and may have tipped something).
That said, as last time NVDA is not speaking fast enough to keep up with me while typing this in NotePad after working in MuseScore and closing it.
I closed NVDA and ReOpened it and now it is now reading what I type as fast as I can type it.
To double check to see if repeatable on my machine:
I just opened muse score again, hit escape, got the edit window, hit alt and then arrowed across the list of menus and hit alt f4 to close MuseScore.
I then opened NotePad and NVDA could not read as fast as I am typing.
I again closed NVDA and ReOpened it and now it is reading what I type as fast as I can type it.
Just a heads up to put in the back of your mind in case you run up against something while analyzing NVDA and MuseScore while deep in the weeds.
Do you know if anyone has ever considered forking a full conversion to a Windows, Visual Studio Managed language?
PS: I will read the manual before setting up my next Navigation test likely dealing with one or more of the File Menu options.
Thanks for the dedication.
Rick USA
In reply to Hi Mark: I opened MuseScore,… by Rick USA
Thanks for the update!
I can't think of any particular reason NVDA's response rate would vary like that. I'm betting you have it speaking much faster than I do, but I don't notice any obvious variation. Do you have the NVDA update that just came out today (or maybe yesterday)? Also, do you have JAWS? Have you tried using MsueScore with the JAWS script we just released?
I can see that we'd ever rewrite MsueScore to make t not cross-platform, and can't imagine any benefit to doing so. There's no particular Qt cannot do everything we need it to do. even if there is some bug or limitation now, it's easier to fix that bug (since Qt is open source too!) than to rewrite MuseScore. And rewriting would solve any of the problems you mention. It's still up to us to decide how to organize the objects in our interface, whether we write that code with Qt or some other library.
In particular, the object tree itself is not a problem, you can view that with Accessibility Insights and see things in some sort of logical organization just as we designed it. We just didn't design it as well as we could have if we were thinking about accessibility when we decided how to lay out the window. We know more now, and can see some ways of reorganizing things. The palette, for example, has a pending change so that instead of over a dozen press of Tab to make it through, it will only take a handful, because we're going to change what activates with Tab versus the arrow keys. I can also see us doing things like having F6 move directly from the Inspector to the Palettes to the Toolbar, or using things like the various shortcuts one typically sees in a web browser to move from heading to heading or from landmark to landmark. Unfortunately, while there is great standardization for such things with regard to the web, there is much less for applications. But we'll figure something out
FWIW, I have an intern right now going through the menus and dialogs to report problems also. We've been more focused on the score itself than the various dialogs, since those should mostly "jsut work", but there are indeed a few controls here and there missing labels or names so they don't read properly, and also a couple of dialogs where for some reason, the entire dialog reads the moment you transfer focus to the first control, thats something we definitely need to look at.
It shouldn't be the case, BTW, that any standard controls act in other than standard ways. In particular, tabbing through the Inspector should never change any of the values of any of the spinboxes, combo boxes, etc. But using the arrow keys certainly would.
In reply to Thanks for your comments! … by Marc Sabatella
Hello,
My question is about Musescore accessibility with Voiceover on macOS. I know really good NVDA suport for Windows, but now I was tried Musescore with Voiceover and it doesn't working so good - for example it is impossible to reading scores note by note - Voiceover don't speak anything. Just let me know, is it possible to make Voiceover suport in next version - Musescore 4, or not?
Btw - congratulations about good job with Windows and NVDA - it will be very good to make the same with MacOS.
In reply to Hello, My question is about… by ttjsxyj9is
Hello, and thanks for the writing! We have been trying to get MuseScore working with VoiceOver, but no luck so far. We're working on some ideas that might help once we get to MuseScore 4, but unfortunately that will not be very soon.
By the way, we do now work with JAWS also, and as of 3.5 which is coming very soon, we will work with Orca on Linux also.
Hi Mark:
I have gone through the initial Setup Process.
In the Preferences Dialog, general tab, there is a Splash Screen Checkbox which is checked by default.
I did a search on the word splash in the edit search box but nothing came up from the manual.
What is this Splash Screen and should I Uncheck it or leave it checked?
i did uncheck the Start Center Checkbox and MuseScore is working as expected - the Main Window comes up after clicking the MuseScore Desktop icon.
Thanks for your Dedication Mark:
Rick USA
In reply to Hi Mark: I have gone through… by Rick USA
You can safely uncheck the splash screen as well. It is a rectangular logo-like image shown during startup. Loading that single image is fast and a visible feedback to non-blind users that musescore is in fact launching.
In reply to Hi Mark: I have gone through… by Rick USA
A splash screen is a purely visual window, usually with a nice graphic and logo, that appears while a program is starting up, to let people know, "we are busy starting up, please be patient, the real main window will appear when we're ready. It's important for programs that take more than acouple of seconds to startup, and MuseScore can indeed take a while if you have large soundfonts installled. To be honest, I can't imagine a good reason to ever want to turn the splash screen off, or why we even have an option to do so. But someone must have wanted it at some point. I would leave it alone. FWIW, as far as I can tell, the splash screen is ignored by screen readers. They don't announce its appearance, they only announce "MuseScore 3" when the actual window appears.
In reply to A splash screen is a purely… by Marc Sabatella
Hi:
I am creating some shorter, personally structured, concise overview documentation for items in the File Menu.
1) In a few words for brevity can you tell me the intended diferences in purpose between [Save As...], [Save A Copy...] and [Save A Selection...]?
2) Do they all use the same default folders when their dialogs come up or is there a setting to use the source folder of a score or selection as the default target folder in some setting someplace?
Finally,
3) do you know of any confusions experienced by clients or gotch yas I might document for raw nubes?
Since I am not likely to get to using these features anytime soon some short input from experienced users would give me a confirmation of what I think I understand about these features after having reading sections of the Handbook and many Forum postings.
Thanks for your dedication:
Rick USA
In reply to Hi: I am creating some… by Rick USA
"Save As" and "Save a Copy" mean what they do in most applications. "Save As" saves the file to a new location and makes that the "current" version (so further "Save" operations go to that new location). "Save a Copy" saves the file to a new location but keeps the "current" version as it was (so further "Save" operations go to the original location). "Save Selection" creates a new score consisting of just whatever is currently selected and saves that to the specified location. As far as I know, they all default to the current folder.
As for common confusions, there are many, but there are also many existing tutorials and other sources of information. I do think a tutorial specifically geared toward blind users would be useful. The "Accessibility" section of the Handbook is great for learning how to work the interface using keyboard and screen reader, but it doesn't tell you much about actually using the program. So a separate tutorial that kind of combined both would be welcome I'm sure. It's something I've been wanting to do for a while, just haven't had the time
In reply to "Save As" and "Save a Copy"… by Marc Sabatella
Hi Mark:
I am currently walking the File Menu and giving a few words on what each menu item does and any comments or notes of interest to a screen reader user.
Your description of the oft confused use of the
Save As and Copy Menu Items was perfect for what I was looking for.
My first task is to describe Menu Items inside each Main Window Menu and what each does and a short comment if beneficial to screen reader users.
When done I will make them available and consider the next level of docs perhaps based on individual activities a user may want to do creating, editing or otherwise working with MuseScore in some fast way to find step by step instructions to accomplish each small task via the keyboard.
What format they will end up in, text, pdf, chem, MS Word, HTML, XML etc... may be up to you folks but for now just trying to make some docs geared twoard the blind and raw beginner is enough to keep me busy for awhile me thinks.
Thanks Again and I hope you dont get mad at what will surely seem like funky questions from me, sigh - sorry:
Rick USA
In reply to Hi Mark: I am currently… by Rick USA
Please do not worry about making us mad with questions! That is the whole purpose of having these forums, and we rely on feedback from users to improve the software!