[Autocomplete] Broken for iOS VoiceOver

[useit-consulting]

trafficstars

• [x] The issue is present in the latest release.
• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

Using VoiceOver on iOS, I am unable to either find or activate any of the suggestions in the list. This is a critical accessibility barrier for visually impaired mobile users.

I can find the input, type text into it and get results, but I cannot set focus on the listbox containing the results. As soon as I move VoiceOver focus from the input field, the listbox disappears. It does not matter if I use explore-by-touch or the swipe-forwards gesture.

See video demo of the problem: https://youtu.be/VfFfFncFcUs

Expected Behavior 🤔

• No big UI changes should happen when I move VoiceOver focus on the page.
• The listbox should stay open when VoiceOver focus leaves the search field.
• I should be able to set focus to the results in the listbox by swiping forwards from the combobox, or by searching with my finger on the screen

Compare with the WAI-ARIA Authoring practices 1.1 example pattern (https://www.w3.org/TR/wai-aria-practices/examples/combobox/aria1.1pattern/listbox-combo.html) which displays the expected behavior:

Video demo of expected behavior: https://youtu.be/8bzbR2vhVtY

Steps to Reproduce 🕹

Steps:

1. Go to https://material-ui.netlify.app/components/autocomplete/
2. Start VoiceOver
3. Activate the combobox and start typing
4. Try to find and activate a suggestion in the list

Context 🔦

This is a critical accessibility barrier with no way to get around it for a majority of visually impaired mobile users. iOS VoiceOver is used by 71% of mobile users according to WebAim Screen reader user survey #8 (https://webaim.org/projects/screenreadersurvey8/#mobilescreenreaders)

Also, this appears to be new behavior. Compare with screenshots in https://github.com/mui-org/material-ui/issues/18191 where the listbox did stay open when leaving the input field and you could find suggestions by explore-by-touch.

I am not sure if there have been any system-level changes affecting this going from iOS version 12 to 13 and 14, or if the component itself has changed. I tried looking at older versions of the component but could not find one that did not exhibit this bug.

Your Environment 🌎

Tech	Version
Material-UI	4.11.0 (and 5.0.0-alpha.11)
React	16.13.1
iOS	14.0.1
iOS Safari	14.0.1
iOS Chrome	85.0.4183.109

[oliviertassinari]

https://material-ui.netlify.app/components/autocomplete/#ios-voiceover? #18191

[useit-consulting]

@oliviertassinari No I'm afraid not. This appears to be a separate issue.

I was able to reproduce it even with disablePortal (see https://codesandbox.io/s/autocomplete-experiment-ios-voiceover-bug-qc8zp?file=/demo.js ) . The listbox still closes immediately after moving VoiceOver focus from the input.

[oliviertassinari]

@fymmot Thanks for the report. I couldn't find a way to reproduce. I can only use iOS on my mac Xcode simulator, which doesn't seem to support VoiceOver.

[eps1lon]

@useit-consulting Could you exactly describe what steps your doing in your youtube video?

Using disabledPortal. I can navigate each item by swiping left and right. I can also activate it with a double-tap.

Actionable steps for us regardless: Set disablePortal as the default. The autocomplete not working on VoiceOver effectively renders it inaccessible considering how popular VO is. The components should be accessible by default. If not there should be a strong rationale for not being accessible by default but at least the initial demo should be accessible for auditing purposes.

[useit-consulting]

@eps1lon @oliviertassinari

I hope this new video makes it more clear:

https://www.youtube.com/watch?v=99g-jEQKirg

First I am swiping right from the input field, expecting to find the list of suggestions. Instead, the suggestions disappear and and the input field is cleared.

In my second attempt, I am using explore-by-touch, dragging my finger from the input field downwards on the screen, expecting to find the suggestions. Again, the suggestions disappear and the text field is cleared.

I'm testing in a sandbox that has the prop disablePortal (https://codesandbox.io/s/autocomplete-experiment-ios-voiceover-bug-qc8zp?file=/demo.js.

The device is an iPhone SE with iOS 14.0.1

[eps1lon]

The device is an iPhone SE with iOS 14.0.1

Works on my iPad with iPadOS 14.1. I didn't think iPadOS has that much difference to iOS.

Let me get a deploy for you that is closer to how the APG combobox.

[eps1lon]

@useit-consulting What I don't understand is why you click again in the textbox after typing. This might trigger a close expectitely. As far as I can tell you can swipe between items directly after you typed. No need to manually click again

[useit-consulting]

@eps1lon

@useit-consulting What I don't understand is why you click again in the textbox after typing. This might trigger a close expectitely. As far as I can tell you can swipe between items directly after you typed. No need to manually click again

It makes no difference, the outcome is the same. If I press the "Done" button after typing to hide the keyboard, the suggestions disappear. Same if I swipe backwards to leave the keyboard.

If you need I can record another video to demo.

[eps1lon]

To clarify: On an iPad I don't have to press "Done" or anything. I can type a letter and then swipe right and left between results. You should never have to actually leave the keyboard. That would go against how a combobox is supposed to be used.

It's not clear whether VO or you are accidentally closing the keyboard (which closes the combobox) or if VO is not recognizing the context.

Could you record a video:

1. what happens when you immediately swipe left and right after typing a letter
2. how does https://www.w3.org/TR/wai-aria-practices/examples/combobox/aria1.1pattern/listbox-combo.html#ex3_label behave

[useit-consulting]

@eps1lon Left-swiping after typing in Mui autocomplete: https://www.youtube.com/watch?v=wP6cTuFTb50

I swipe left repeatedly until I leave the keyboard. The keyboard is then closed automatically. The suggestions are closed and the input is cleared. It was not in the video, but If I instead had swiped right after typing a letter, I would go to the next key in the keyboard.

---

(I tested several of the examples. But I believe WAI-ARA example # 1 is the most relevant one here, no? Because like MUI autocomplete, it has no automatic selection from the list, you need to click manually on a suggestion. )

WAI-ARIA listbox example 1 - swiping left after typing. https://www.youtube.com/watch?v=RAIBoIiBPN8 Note that this is not 100% perfect because you end up somewhere else on the page when the keyboard closes. But once you get back to the autocomplete, the listbox is still open and you can focus on the suggestions.

WAI-ARIA listbox example 1 - explore-by-touch. https://www.youtube.com/watch?v=jGrwQjhqsIw No problems at all to find suggestions when using explore-by-touch on the suggestions list.

---

WAI-ARIA listbox example 3 - swipe left after typing. https://www.youtube.com/watch?v=ruvmIDAw2iU The autoselecting example # 3 is confusing and a bit buggy. It would be difficult for a blind user to understand if they made a selection or not. Also, I was not able to re-open the autocomplete again at the end of the video.

But as I said, believe example # 1 is the most relevant one because it does not auto-select a suggestion.

[oliviertassinari]

Actionable steps for us regardless: Set disablePortal as the default. The autocomplete not working on VoiceOver effectively renders it inaccessible considering how popular VO is. The components should be accessible by default. If not there should be a strong rationale for not being accessible by default but at least the initial demo should be accessible for auditing purposes.

@eps1lon react-select doesn't portal by default and has created some friction: https://github.com/JedWatson/react-select/issues/2398. select2.org portals by default.

It seems that our users care more about features than accessibility and that desktop is a more prominent use case, so it might be a case where iOS support should be optional if there is any significant downside from a feature perspective. We could always try a different tradeoff and see how it goes.

In terms of downsides, I'm aware of 1. scroll containers/dialog with overflow, flex container.

[eps1lon]

@eps1lon react-select doesn't portal by default and has created some friction: JedWatson/react-select#2398. select2.org portals by default.

And what is this supposed to tell us?

It seems that our users care more about features than accessibility

I don't care what our users care about if they don't care about their users. The goal is to provide useful primitives that can be leveraged for a good UI. Not provide any primitive for the UI any person wants to build.

and that desktop is a more prominent use case, so it might be a case where iOS support should be optional if there is any significant downside from a feature perspective

So you've changed your mind when we're discussing a11y by default in https://github.com/mui-org/material-ui/pull/22382#issuecomment-698880283? Then we need to re-open this discussion and decide how we want to communicate this. Because having a "how to make a11y" at the bottom is not ok. You don't write at the end of your manual how you can use an appliance.

[oliviertassinari]

And what is this supposed to tell us?

@eps1lon I think that we can use it to understand the downsides of not using the portal (a.). react-select is enough used to have developers report the issue that comes with it. For instance, we can leverage it to add a documentation section about the downsides of using disablePortal.

The goal is to provide useful primitives that can be leveraged for a good UI.

Agree with this objective! It's about allowing developers to do a better job. From what I understand, it's not trivial that trading against iOS support against (a.) is what maximizes this direction. So far, the problem was raised twice by the same entity (company) #18191 and #22956. I guess it's about how much (a.) is a pain compared to iOS voiceover.

What would be wrong with saying that due to a lack of accessibility capability of iOS, there is no default support for it, but however we provide this opt-in option with this list of downsides (a.)? We could open an issue on the iOS Safari bug tracker, so it gets fixed in the future (if not existing already). From what I understand, it doesn't break the AA or AAA objective https://github.com/mui-org/material-ui/pull/22382#issuecomment-698880283.

In any case:

• I'm not against trying without the portal by default ("We could always try a different tradeoff and see how it goes")
• I think that it would be great to come to the bottom of #22956 before changing the documentation.
• I think that once we come to the bottom of #22956 documenting the downsides of no portals / upsides of using a portal would be great.

[eps1lon]

react-select is enough used to have developers report the issue that comes with it.

I'm not following. Did you search the react-select issues and didn't find one reporting a11y issues? If so and you didn't find one: What do you think can you deduce from it? Someone not reporting a11y issues does not imply it has no a11y issues. Similar to survivorship bias.

From what I understand, it doesn't break the AA or AAA objective

No, because it technically breaks A i.e. it isn't operable at all on VO on iOS (70% usage).

What would be wrong with saying that due to a lack of accessibility capability of iOS, there is no default support for it

70% of users on mobile use VO. Saying that it has a lack of a11y capabilities because it doesn't support aria-owns is a bit harsh.

I think that once we come to the bottom of #22956 documenting the downsides of no portals / upsides of using a portal would be great.

Yes! I think that we should do this regardless because we don't really document anywhere why you might want to disable the portal in general.

[useit-consulting]

@eps1lon @oliviertassinari I see the discussion expanded a bit beyond the original scope.

Just wanted to point out that the issue we are discussing is separate from the portal problem. I can reproduce it even with the disablePortal prop, as you saw in the videos and the CodeSandbox.

And as opposed to the aria-owns/portal problem, this is a complete barrier that I don't see any way of getting around for a VoiceOver user.

Can we try to figure out what is causing the auto-closing of the listbox when VoiceOver focus leaves the keyboard or the input, and what makes the Mui autocomplete different from the WAI-ARIA authoring practices examples where this does not happen?

[eps1lon]

Just wanted to point out that the issue we are discussing is separate from the portal problem.

Not really. It's just not the root of the problem. In the end you still will have to disablePortal on VO. So the issue has two problems:

1. VO needs disablePortal
2. VO on iOS is bugged even with disablePortal while working fine on iPadOS

[eps1lon]

@useit-consulting Could you check-out https://w3c.github.io/aria-practices/examples/combobox/combobox-autocomplete-list.html and tell me if this is working for you?

[useit-consulting]

@eps1lon Apologies for the delayed reply.

That example does not work with my iPhone. The behavior is almost exactly like the Mui autocomplete:

• Listbox popup disappears when I swipe away from the input
• Listbox popup disappears when I finish typing and close the keyboard with the "Done" button
• Listbox popup disappears when I try to explore-by-touch, unless I place my finger directly on a suggestion

Let me know if you need a video to demonstrate

[useit-consulting]

@eps1lon I've done some more digging today and I believe I know the cause of this issue.

There was a change in default VoiceOver behavior that came with iOS 13 and iOS 14. Moving VoiceOver focus will now trigger keyboard focus and keyboard blur events by default. This should be why the listbox closes when searching the screen.

So I think we can try adding an option/prop to keep the listbox open on input blur events and see if that solves it

[eps1lon]

That example does not work with my iPhone. The behavior is almost exactly like the Mui autocomplete:

Could you open a new issue on the apg repo (https://github.com/w3c/aria-practices/issues/new)? It'd be interesting to get their input on why this isn't working. I seem to recall them saying we should test their implementation before using. But I think their implementation should at least work in the most popular SR for mobile devices.

[gregnb]

@eps1lon I've done some more digging today and I believe I know the cause of this issue.

There was a change in default VoiceOver behavior that came with iOS 13 and iOS 14. Moving VoiceOver focus will now trigger keyboard focus and keyboard blur events by default. This should be why the listbox closes when searching the screen.

So I think we can try adding an option/prop to keep the listbox open on input blur events and see if that solves it

@useit-consulting Is there any thread you can point me to that highlights this change in IOS13 -> IOS14 ? bug ticket? would be helpful for my own tracking purposes.

i am also facing this issue you describe in the thread whether it's W3C's example or MUI's autocomplete. For my own project I'm leveraging react-select and seeing the same issues with that.

[useit-consulting]

@gregnb I've tried to find some concrete info, but so far I only have second-hand and anecdotal information. It was mentioned in a thread in the a11y Slack workspace (https://web-a11y.slack.com/)

I have logged this issue in the WAI-ARIA authoring practices GitHub page. Let's keep track and see what they say about it. https://github.com/w3c/aria-practices/issues/1619

[useit-consulting]

Still waiting on updates and feedback in the WAI-ARIA thread.

But in the mean time @eps1lon , would you consider a temporary workaround that closes the autocomplete on keyboard tab and outside click, instead of using onBlur?

I believe that would eliminate this issue, without affecting the UX for keyboard or mouse/touch users. See https://github.com/w3c/aria-practices/issues/1619#issuecomment-730469880

[eps1lon]

But in the mean time @eps1lon , would you consider a temporary workaround that closes the autocomplete on keyboard tab and outside click, instead of using onBlur?

I think you can implement that locally. I'd rather not immediately use the very first workaround we know of. These changes are always scary since we have poor e2e test coverage for these types of interactions. Especially since it's not clear whether this is a screen reader bug and how likely it is that it'll get fixed.

[useit-consulting]

@oliviertassinari @eps1lon

This bug has now been fixed in the WAI-ARIA 1.2 Combobox pattern (using the previously mentioned workaround with tab and outside click listeners). See https://github.com/w3c/aria-practices/pull/1699

Could you have a look at implementing the same change for the MUI autocomplete?

[oliviertassinari]

@useit-consulting I had a look at the solution, it doesn't seem something we should replicate in the same form here. Did they consider a simple debounce on the focus/blur event? It's something we do in the tooltip or speed dial.

[useit-consulting]

@oliviertassinari Hmm, I'm not sure (I don't have any more info than what is in the GitHub thread). But it sounded like cookiecrook from Apple was suggesting something similar.

If you put together a build with debounce, I can test it with VoiceOver for you and see if it does the trick

[useit-consulting]

@oliviertassinari Do you have any update on this?

As I said before, this is a critical barrier for mobile screen reader users. And it can also be a legal liability for any organization using the autocomplete.

It is currenty stopping one of our clients from using the MUI autocomplete in their production app.

[niklasburman]

@oliviertassinari Do you have any update on this?

As I said before, this is a critical barrier for mobile screen reader users. And it can also be a legal liability for any organization using the autocomplete.

It is currenty stopping one of our clients from using the MUI autocomplete in their production app.

I can confirm that, we have issues with the component as described. Would be really great with any status regarding the issue. /Thanks

[oliviertassinari]

Related to #25365.