FreedomScientific
JAWS is not reading MathML in a clear, unambiguous manner
Summary
For aural only consumers of math content, the mathematics must be read aloud in a clear, unambiguous, and standardized manner for the mathematics to be comprehended accurately. As of June 6, 2021 JAWS is failing to do so. The result is that consumers of audible only math content are being mislead and disadvantaged when using the JAWS screen-reading software's native mathematics support.
Expected result
There is a formality to mathematical expressions that communicates the relationships between subcomponents contained within mathematical expressions. Standards have been developed (like MathSpeak Grammar by Dr. Nemeth creator of the Nemeth Braille code) for accurately and unambiguously speaking mathematics. The expectation is that JAWS would either follow a recognized standard or have developed one that is capable of speaking mathematical content clearly and unambiguously.
Actual result
JAWS fails to speak even basic math unambiguously from simple MathML content regardless of the JAWS punctuation verbosity settings.
Example
Here is a test page with six example pairs of simple mathematics. The mathematics in each pair are obviously distinct mathematically and yet JAWS will speak the two paired MathML examples identically: JAWS not reading MathML in a clear, unambiguous manner - https://brichwin.pages.iu.edu/tests/jaws_Reading_MathML_ambiguously-2021-06-07.html
Here is a codepen version if that works better: https://codepen.io/brichwin/pen/WNpyyva
Additional Information
JAWS version and build number
2021.2105.53 ILM
Operating System and version
Microsoft Windows 10 Home Version 10.0.19042 Build 19042
Browser and version:
Firefox 89.0 (64 bit), Chrome Version 91.0.4472.77 (Official Build) (64-bit)
Note: The examples provided include the JAWS speech history of how each bit of mathematics is being spoken as well as the MathSpeak Grammar for each example (as rendered by MathJax). This was meant to provide examples of how the mathematics in the examples could be spoken unambiguously.
Tested today, 2021-10-11 and confirmed that this issue of JAWS not reading even basic algebraic math correctly is still present in the latest version of JAWS. Any updates on the issue?
N. Soiffer's MathCat project is nearing a ready state. It would be awesome if JAWS adopted using MathCAT to read MathML content instead of trying to roll such an ambitious project on your own. Also, if MathCAT was adopted by multiple AT vendors, then a standardized method of reading mathematical content would greatly increase the easy of comprehending audible and braille math due to the consistency.
Please give utilizing MathCAT some serious consideration.
Knock knock. Any movement on this? With the new school year starting soon, it would be nice to have JAWS and math working.
I'm working on thesis which could help in this use-case. I will leave more information, when i have results
FYI Current JAWS documentation for MathML support
MathML
MathML is a language for including mathematical content on web pages, enabling browsers to present math and science equations and formulas as they would appear on paper. JAWS provides spoken, natural language descriptions of MathML expressions in Edge, Chrome, and Firefox using the same terms a teacher would use to explain them in a classroom. Students will benefit from hearing JAWS describe mathematical expressions as their teachers and peers do. If a refreshable braille display is connected, JAWS also shows math expressions on the display using the Nemeth Braille code, a widely used system for reading math in braille. In addition, math expressions are also displayed with highlighting in the Math Viewer as they are spoken so sighted teachers can assist students that are using JAWS to study math.
Note: Since Nemeth Braille is a system exclusive to English, MathML content can currently only be shown in braille if JAWS is configured for one of the English Contracted Braille output modes.
Once JAWS is installed, you can immediately begin reading math and science content. You do not need to install any additional third-party software applications or plugins to use JAWS MathML support.
When you encounter MathML on a web page, JAWS describes the expression, followed by the message "math content." If you are using a braille display and JAWS is configured for Contracted Braille, the message "Math" is displayed followed by the expression. If the expression is too long to fit on the display, you can use the panning buttons on your display to view the rest of it. Otherwise, if JAWS is set for Computer Braille, the message "Math" is displayed followed by a message stating that Contracted Braille must be enabled to view math content. Pressing ENTER while focused on the math content opens the Math Viewer where you can explore the expression in greater detail in both speech and braille by doing the following:
- To move between parts of the expression, use the LEFT and RIGHT ARROW keys.
- To focus in on a partial expression, press DOWN ARROW.
- To return to the whole expression, press UP ARROW.
- To read all the partial expressions at the current nesting level, press INSERT+UP ARROW.
- To read the current partial expression, press NUM PAD 5 or INSERT+NUM PAD 5.
- From within the Math Viewer, press ESC or ALT+F4 to exit.
Additionally, the contents of the Math Viewer are displayed visually on the screen. As the equation is navigated, the current expression or partial expression is highlighted so a sighted assistant can easily tell what part of the expression is being spoken.
When using a braille display, the entire expression remains on the display as you use the ARROW keys to navigate parts of the expression. However, the cursor (dots 7 and 8) moves to the start of the partial expression that gains focus. Pressing a cursor router over a term in the expression will immediately move the cursor to the specific location and cause JAWS to speak and visually highlight the partial expression as if you moved to it with the ARROW keys.
If you are viewing an expression containing a table, JAWS indicates in both speech and braille the number of columns and rows in the table. When the table gains focus, press ENTER to go into Table Navigation. In Table Navigation, the UP and DOWN ARROW keys move between rows while the LEFT and RIGHT ARROW keys move between columns. Press ENTER on a table cell to enter Equation Navigation so you can explore the current cell. Press ESC to return to Table Navigation, and then once more to return to Equation Navigation for the whole expression.
On certain sites, such as Wikipedia, math is displayed as a graphical image which is not readable by JAWS. However, in the case of Wikipedia, you can specify that the MathML that generated the graphic is included directly in the HTML of the page. Making this change requires you have a Wikipedia account. Creating an account is free, but the process includes a captcha which will require sighted assistance, or the WebVisum add-on for Firefox, to complete. Once logged in, go to your advanced preferences and select "MathML with SVG fallback" to render math in HTML.
Visit the MathML Examples page To experience JAWS and MathML support in action.
I heard a somewhat reliable rumor that the developers are working with Neil Soiffer to have the option to use MathCAT in JAWS. MathCAT is essentially the new replacement for the aging MathPlayer. See: https://nsoiffer.github.io/MathCAT/ for details.
This will be amazing as MathCAT has many preferences and end users can chose from a variety of options including ones that create accurate / unambiguous speech that makes every attempt to be as concise and as friendly to listen to as possible. It also includes some well developed math expression navigation features that can be used without having to launch a separate "expression explorer".
@brichwin I can say that there are a substantial number of active work items related to MathCat in the internal tracker, so the rumour is reliable. 💯