Provide screen-reader-friendly Vertical Layout

[humenda]

Zulip-terminal uses a "TUI", i.e. vertical splits of the terminal, to provide an intuitive user experience. Unfortunately, screen readers read, on the command line, the screen from left to right, hence mixing all vertical splits into a single one, when navigating the screen line by line. This renders the UI unusable.

For screen readers, a vertical split works much better (compare irssi, mutt, ...). If the different windows could be displayed from top to bottom, it would make this application accessible.

Screens are usually wider these days, so this would likely be an optional feature, but I would appreciate if this could be considered, as this would allow me to use Zulip.

[neiljp]

@humenda Thanks for reporting this!

My initial query on how to resolve this was how the content in the sidebars might be shown, if it would be above/below the other content, not in sidebars. Aspects such as the list of streams, topics and users naturally fall into lists after all :)

There is currently an autohide mode that mainly shows the central message list. Upon moving left or right, the left or right columns are temporarily overlaid on top of the message list until an element is selected, or upon moving back into the central message list column. Currently this does both overlay the content over the message list, and show small 'tabs' in the main view to hint that the left/right columns are present.

Therefore, one possible solution might be to have a less TUI-heavy autohide mode, where moving left/right switches into showing only the content of the left/right side bars, and the hinting 'tabs' in the message list could also be hidden.

Would that design work?

[humenda]

My initial query on how to resolve this was how the content in the sidebars might be shown, if it would be above/below the other content, not in sidebars.

If the result is that per "physical" screen line, there is just one view, i.e. content line, then this would indeed help.

Therefore, one possible solution might be to have a less TUI-heavy autohide mode, where moving left/right switches into showing only the content of the left/right side bars, and the hinting 'tabs' in the message list could also be hidden.

What are "hinting tabs"? Without knowing the actual effect, this might be a workable solution. I would anyway rely on one view/UI frame at a time, so the hiding functionality sounds reasonable. Justlike the tabs on Android where you swipe left/right.

Thanks!

[neiljp]

A left hinting tab looks a little like like below, with message content to the right of it, which is why I thought to avoid it:

< |
S |
T |
R |
E |
A |
M |
S |
< |

The design I described above, without tabs or the left/right columns, would essentially make each of the main columns 'full screen' and have no need for vertical line symbols.

Other design elements off the top of my head which would be useful to know if they would impact screenreader usage:

• we have 'popup' windows, which are overlaid over the other UI parts (eg. the message list); need to be 'fullscreen'?
• there are various horizontal lines to separate sections and indicate titles
• message sender/date-time/starring is shown in a line above the message content
• message content is offset by a vertical bar, with a possible 'EDITED' to the left
• a full message is selected at once, over multiple lines, and up/down motion selects previous/next messages; I'm not sure how a screenreader would pick up on the range of lines being considered as one element?

This may be too detailed, but in general, a message element can currently look a little like this:

 [stream symbol] stream-name [right triangle] topic-name [horizontal line]
[sender-status symbol] sender-name      [message-sent date/time] [optional star]
[possible EDITED] [block] [first line of message content]
                  [block] [second lines of message content]

Footlinks and reactions are shown below a message, though the former are configurable, and the latter are already in text form which I suspect is useful - compared to emoji?

Re your original point about screen width, were you meaning that your screen is narrow? I didn't quite understand that.

[humenda]

A left hinting tab looks a little like like below, with message content to the right of it, which is why I thought to avoid it:

< |
S |
T |
R |
E |
A |
M |
S |
< |

Yes, this is cumbersome to read, indeed.

The design I described above, without tabs or the left/right columns, would essentially make each of the main columns 'full screen' and have no need for vertical line symbols.

Yes.

Other design elements off the top of my head which would be useful to know if they would impact screenreader usage:

• we have 'popup' windows, which are overlaid over the other UI parts (eg. the message list); need to be 'fullscreen'?

Yes, I would say so. Note though that horizontal splits aren't a problem, as the screen reader operates per line (as a simplification). However, if we would have a horizontal split, the screen reader would obviously only see changes at the current cursor position, missing out on updates in the other split, unless explictly asked for.

• there are various horizontal lines to separate sections and indicate titles
• message sender/date-time/starring is shown in a line above the message content
• message content is offset by a vertical bar, with a possible 'EDITED' to the left

I assume it is the example that you provided below?

• a full message is selected at once, over multiple lines, and up/down motion selects previous/next messages; I'm not sure how a screenreader would pick up on the range of lines being considered as one element?

There is a mode to navigate the screen, so if up/down selects messages, I would expect the cursor always to be positioned at the first charcter of the message and then navigate the screeen to read the rest. personally I do prefer being able to navigate messages line-wise, as in Vim/w3m, but that's not an a11y requirement. The screen reader provides functionality to emulate this.

[stream symbol] stream-name [right triangle] topic-name [horizontal line]

What would be a stream symbol?

[sender-status symbol] sender-name      [message-sent date/time] [optional star]
[possible EDITED] [block] [first line of message content]
                 [block] [second lines of message content]

When reading a bunch of messages, it is often most important to read the name of the sender and the message. Is this configurable? If not, I would probably propose:

name: message | meta info

(regarding the order, not as a template for spacing, formatting or punctuation)

Footlinks and reactions are shown below a message, though the former are configurable, and the latter are already in text form which I suspect is useful - compared to emoji?

Yes, they probably are. My screen reader actually interprets some unicode symbols, but a text equivalent is probably more portable.

Re your original point about screen width, were you meaning that your screen is narrow? I didn't quite understand that.

A braille display has commonly 40 or 80 characters and a single line. I just wanted to make you aware that I see a line at a time, so am lacking context. The amount of characters on a braille display don't really matter.

[neiljp]

Apologies for not getting back sooner, let me see if I can address your points:

• the best design would seem to be to switch between various full-screen windows: the left panel (streams/topics/views), messages panel (center), right panel (users), and then overlay popups as full-screen too
• regarding line-characters, no vertical/corner lines
• the message example is as I posted
• stream symbols are currently # (public stream), P (private stream) and another for web-public (no account required, though ZT doesn't support browsing like this); symbols are just mapped in a source file however, so they could be customized, and we've discussed 'theming' them for those with or without emoji - or with very basic terminals!
• the current message layout is based on the web design; it varies but I often see long Zulip messages, compared to the short ones on IRC, which I suspect is why the meta info is used as a header
• there are also 'conversation' headers in interleaved views like all-messages, or all direct messages, or a stream view, to describe each conversation (topics, recipients); you need not see those depending on the view you look at
• reactions, or emoji in messages, are shown as eg. :smile: for :smile:

For implementation, I suspect we could try the full-screen aspects first, to see if that makes it testable for your system to read?

The messages seem like the other tricky aspect to consider after that.

[humenda]

• the best design would seem to be to switch between various full-screen windows: the left panel (streams/topics/views), messages panel (center), right panel (users), and then overlay popups as full-screen too

Yes, though it would be ok to have a horizontal split. At least pure command-line screen readers have commands to remember screen positions and to jump the the top or bottom. If it is visually appealing, two windows on top of one another aren't problematic. Compare Irssi or Mutt or any other command-line application with a status bar.

• regarding line-characters, no vertical/corner lines

Yes, I would say so.

• the message example is as I posted

Fine. The message format looks akin to the one of Irssi.

• stream symbols are currently # (public stream), P (private stream) and another for web-public (no account required, though ZT doesn't support browsing like this); symbols are just mapped in a source file however, so they could be customized, and we've discussed 'theming' them for those with or without emoji - or with very basic terminals!

Any is fine. As @, #, etc. are spoken aloud and displayed on a braille display without issues, they could be left as-is.

For implementation, I suspect we could try the full-screen aspects first, to see if that makes it testable for your system to read? The messages seem like the other tricky aspect to consider after that.

Agreed. Thanks a lot for taking the time to implement this!

[neiljp]

Thanks for clarifying the situation, it looks like we have a plan on how to take this forward :+1: I've labelled this as help-wanted, since the provisional plan could help someone get started.

This looks a little like other issues I've been browsing, such as #1170 (collapse sidebars)

There was also various related work to eg. scale the UI with terminal width, so I've labelled these with a new area to help anyone working on this look at similar ideas.