element-hq
accessibility: use talkback actions as alternative to the bottom sheet on long-press
Your use case
What would you like to do?
I would like to have an easier way to access message actions, such as react, reply, etc
Why would you like to do it?
Screenreader users, especially on mobile, are inconvenienced a lot by UI clutter. For example, having to swipe right three or so times to get to the next message in the timeline is a problem, so is having to long double press, then wait for the timeout to pass, only then finally being focused on the bottom sheet with options
How would you like to achieve it?
For this, accessibility actions are the best fit for what I'm describing.
The screenreader knows how to use them, and more specifically, it emulates better what users have had on ios for a long time for decluttering UI, namely the rotor, where elements are a single swipe targets and, instead of each option being swipe targets after that element, they are incapsulated in those actions one could reach by spinning the rotor to actions and then flicking up and down to select the required action, then double-tapping to confirm.
For more information, see this part of an article from android developer documentation
Have you considered any alternatives?
well, the only alternative is using swipe targets like we did up to this point, with the bottom sheet activating on long press. While better than the every option is a swipe target method, it still requires waiting around for the timeout to pass, which could be frustrating if you have to do an action to afew messages, not just one per minute or so
Additional context
No response
Are you willing to provide a PR?
No
@albertotirla Are you sure duplicating all the bottom sheet actions as accessibility actions is the way to go? Bottom sheet is already accessible, using double tap and hold to open the sheet is a common feature thus I think accessibility actions should be considered for the actions that are now hidden from screen reader users as a result of optimising timeline navigation such as read receipts and devices verification. A compromise I think would still be viable might be an accessible action sayng more actions that opens the sheet as an alternative.
Hello, I'm looking through #4579
And I think we definatelly need to use accessibility actions in order to provide screen reader users access to these hidden features:
- Alternative of clicking the inReplyToDetails so the timeline scrolls to that position
- Access read receipts info as the accessibility action label and action that expands the read receipts
- Encryption state and ability to call ShowShieldDialog
- Consider also being able to expand reactions (I'm not sure if this is accessible already). Thinking more about reactions I think it's already possible long clicking individual reactions to reveal who has posted them and clicking the reaction to send the same reaction. Transforming reactions to accessibility actions for screen reader users would thus degrade the functionality as there is just single action and we'd like to preserve both these actions, so perhaps leaving reactions the way they are today would be a better choice.
- Expose room and user mentions as accessibility actions so screen reader users are able to click on pills. By default talkback and other screen readers can identify text spannables e.g. urls and similar. Since we are using contentDescription for the content I guess this feature is lost. Or perhaps this might be a compose issue I am not sure and I am unable to investigate this.
- Consider providing more polished experience for voice messages, video / audio / files by hiding the inline content that is a single clickable from screen reader users perspective anyway and expose the play / pause, open as the first accessibility action in the list of actions. This is not final decision as if there are multiple controls such as media slider which is now hidden from screen reader users making this into a parent item accessibility action does not make sense. Preferrably it would be best making the slider also accessible. I am yet not sure how this part is behaving for images, videos and other file types. If you can kindly describe this part perhaps we can be able to polish it after some discussion. For example I can see TimelineItemImageView and TimelineItemVideoView are not clickable if screen reader is detected however I can't imagine my-self if this is the part of timeline item or if it's opened as a seperate sheet.
- Consider adding accessibility action labelled more actions to the end of the actions list which opens the bottom sheet as a lighter alternative to the long clicking.
- Find a way on how to expose the fact grouped events view is expanded or not. Perhaps just prepending a word closed or opened to the groupedEventsTitle would be fine and easy to implement I guess.
about whether this is required, as I mentioned in my initial comment, it's not required per say, but then what whatsapp and similar does on ios isn't technically required either, it's just that companies have been incouraged to do this by apple for much longer, enough so that it's now basically a standard or an expected thing. The sheet is there for sighted users, and there is indeed some duplication, but I suppose swift UI makes that slightly easier and we don't have those things because actions came much later over here.
For examples of how much this helps, on both platforms, and how even a huge amount of actions in the menu helps us rather than hindering us, check out the following applications:
- whatsapp, on ios: yes, they walked that back a bit and now some messages are multiple tiles, but still it can be seen how smooth it still is
- the mona mastodon client, ios: here, accessibility actions practically dominate the field, and it's getting so complicated that people needed tutorials to learn to use them. Even so, imagine all that behind a bottom sheet, or several, or in-band, all those being buttons after the initial message tile. Yeah no, no way
- tusky for android: again, messages are one tile, and yes, double-clicking on them opens a threadded view which is expected, not every single thing has to be an accessibility action, sometimes a simple double-click is fine. Speaking of double-click, you can customise what talkback says the action is when you double-click, also presented in that android developer documentation I linked in my first comment
For a very bad example of this, as well as what could happen if we don't use actions, check out fedilab for android. Visually it might look nice perhaps, but I know for a fact that it's inefficient from an accessibility standpoint. Every action possible on a post is next to that post, making navigation with talkback extremely arduous. Yes, I'm aware that commentary screenreader has some ways of skipping those and going to the next list item, but most people use talkback, so if we optimise for that, we optimise for all of them, given talkback is sort of the most common denominator in this space.
So now, about how this would work more precisely. I'm thinking that why not, we can have an accessibility action that just says open share sheet, and that's fine when we don't know whether we want a certain thing to be in actions, it's like an Other or Unknown variant of a per-program error enum, where we shuv all the errors we don't explicitly handle or care about. Yes, that's fine, but it shouldn't be the primary way of thinking here. In particular, this is what I think should be handled by actions, at a minimum, perhaps both on android and ios, but I only have android and that's what my comment is about:
- reply
- reply in thread when that'll be a thing
- react, which opens a reaction dialog
- see reactions, important, different from the react action, and this is only available if the message already has a reaction or more
- edit, where applicable
- share where applicable
- see replied to message where applicable, aka jumps in the timeline
- links, which opens an action with all links. In particular here, pils are seen as links too, so are room aliases, so are links to individual messages, be those in the current room or another one, so are regular links obviously. Then, when one clicks on a pill or room alias, one gets the appropriate ui, in a dialog, again, not everything has to be an action
- view profile or however you want to call that: that opens a dialog like we have now when clicking on a pil, but for the person who posted the message. As much dialog reuse as possible is incouraged here, just the bottom sheet should get some more of a trimming
- more actions: this opens the bottom sheet, and importantly, without the things which are available in accessibility actions for that particular message, to avoid clutter. However, of importance here is that the normal sheet should open on long-press. If that's not possible, just opening the sheet here is good enough
About showing read receats, yes it does, but in another tile from the message itself, that goes for messages being replied to as well. Kind of annoying when I want to go between messages as fast as possible.
O yeah, btw, on ios, some people I know regularly complain about the lack of rotor actions, the only action that exists in there is essentially what we're talking about now as more actions. For ios, that's pretty much a no go, so the research conducted in this aria will probably help there too.
About the media player and so on, if opening a dialog when one double-taps on the media message tile can work, then it should, because again...obvious things can be done with a double-tap if there isn't more than one thing to do, so in this case opening the player and so on in a different modal dialog on double-tap is reasonable and no actions are required here.
Does that make sense so far?
Hello,
I have looked at it from this app perspective while I think you took more general approach by looking at similar apps and trends on various platforms as well. As long as the app remains accessible I agree with you. I don't want to complicate things even further by seperating us as a11y advocates with differing opinions.
Huge thanks for bringing it up and for thorough explanation.
Greetings
Peter
Hi, are there any new updates or thoughts on this matter? I can very much agree to the request and think it would be great if those actions could be added for convenience of blind users. As mentioned above, even other messaging apps such as WhatsApp added those for Android.
I would at least like to know if waiting for some actions resulting from this issue is enough or if it's better to split this into smaller requests. For example I am now missing ability to handle read receipts with a screen reader running and from my perspective it's a regression as it used to work in the past although the timeline presentation was not as polished as it's today.