Clearer error messages for LMS users

[nairiboo]

LMS users encounter a lot of error messages

Problem Statement: Looking back at the November Support report, you can see this trend in the percentage of tickets based off user reported error messages: June 2021 - 10% June 2022 - 17% July 2021 - 5% July 2022 - 27% August 2021 - 1% August 2022 - 23% September 2021 - 2% September 2022 - 28% October 2021 - 7% October 2022 - 24% November 2021- 16% November 2022 - 15%

Some of this is likely due to the fact that more of our error messages make it easy to make a support ticket, but some of it is due to the design that we expect users to run into these messages and read them and act on them. The reality is that users often don’t read errors, or understand them when they do read them.

Rationale: Error messages to users don’t necessarily mean “here’s a misconfiguration you can fix”. In my experience error messages to users mean “this product is having a problem and/or is broken”.

Impact: The impact is based on user perceptions but it’s a bad look for instructors to hear from their whole class that “Hypothesis is broken”, which seems to be one of the messages that gets sent around.

[mkdir-washington-edu]

@dwhly Per our 1:1 we had discussed a general issue to address the high-level issues with error messaging. I think the above is a good start so I'm going to add some thoughts via comment.

Some related issues:

• https://github.com/hypothesis/product-backlog/issues/1475
• https://github.com/hypothesis/support/issues/41
• https://github.com/hypothesis/product-backlog/issues/1271 <- unsure if this is still an issue
• https://github.com/hypothesis/lms/issues/5497

We should also look at this related issue, I believe it's solved and should be closed

• https://github.com/hypothesis/lms/issues/766

The problem here can be broken down this way:

• Many of the errors users run into are only fixable on their end. Instructors need to launch copied assignments, students need to update browsers, instructors need to add students to groups, etc.
• Our current strategy is to show the user an error message that explains what likely needs to be done to fix it, including reaching out to the course professor, etc. These error messages also let the user create a support ticket, which may be good, because...
• Users tend to not read the errors and instead make support tickets, meaning we then have to ask them who their prof is, and email the prof on their behalf to get the issue fixed.

Solutions? One possible way forward is just to make the error messaging more accessible to your average student. The support team could help with crafting these messages.

We should also maybe look at the larger issue of how this works. If when students encountered an error message the instructor was automatically notified in some way that might be better, and if the support team could proactively tie a logged error to an assignment, and subsequently to successful launches of that exact same assignment, we could better track when there are systemic issues and when they've been fixed.