Bare-bones iOS FKA implementation

[LongCatIsLooong]

trafficstars

A bare-bones implementation for iOS focus engine support, to enable basic full keyboard access (FKA) (except for scrolling which will be implemented in a different patch).

Partially f1xes https://github.com/flutter/flutter/issues/76497

https://github.com/user-attachments/assets/427db87e-cc15-483a-85a1-56bf1c02c285

On iOS 15 and above, FKA, if enabled, always consumes relevant key events, so the Flutter framework can't see those key events as they won't be delivered via the UIResponder chain (https://developer.apple.com/documentation/uikit/uikeycommand/3780513-wantspriorityoversystembehavior). This patch provides the basic focus-related information to the iOS focus engine, derived from the information already available in the accessibility tree, so the iOS focus engine can navigate the UI hierarchy and invoke accessibilityActivate on the current focus when the user presses the space key.

This at the moment seems to be the best option:

• There doesn't seem to be a way to reliably prevent FKA from consuming the key events and that seems to be by design.
• The user can remap the FKA keys in iOS system settings, but that key mapping isn't available to apps, so even if the framework can get the key events it won't be able to honor custom key maps.
• When FKA is on, -[FlutterView isAccessibilityElement] is called without user interaction (presumably it's called when the view appears), so when the user interacts with the app using FKA, it's likely that the accessibility is already enabled, we don't have to worry detecting whether FKA is on (at least for now).

Scrolling using FKA currently does not work despite FlutterScrollableSemanticsObject conforms to UIFocusItemScrollableContainer. setContentOffset: must be implemented using a new API that informs the framework of the new contentOffset in the scroll view. accessibilityScroll does not work because it scrolls too much in most cases, and it tells the framework "how much to scroll" instead of "where to scroll to" so it tends to be jumpy.

What happens on iOS versions earlier than 15

I couldn't find iOS 14 runtime for simulators in xcode 16 so I couldn't test it. But since the key events will be delivered to the framework first regardless of whether FKA is enabled, the framework is supposed to handle keyboard focus navigation even when FKA is on.

Pre-launch Checklist

• [ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
• [ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
• [ ] I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
• [ ] I listed at least one issue that this PR fixes in the description above.
• [ ] I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
• [ ] I updated/added relevant documentation (doc comments with ///).
• [ ] I signed the CLA.
• [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[sethladd]

cc @jmagman for iOS and any context, @ericwindmill and @RedBrogdon (for samples), @hellohuanlin (for add2app)

No coding action required. However, any context and thoughts on if we want to maintain this sample? if not, we should remove from our docs.

[hellohuanlin]

I think I discussed with Andrew before - iirc news feed repo has been archived. It was only created for an event. So no point to update it for iOS 18.

[ericwindmill]

There are currently one or two iOS samples in the pipeline. One just landed, and (I think) someone intends on adding a second. @Piinks can give you more details.

With that in mind, do we need this or additional iOS samples? In general, I would like to shoot for fewer apps that show off more features than vice-versa.

[Piinks]

There are currently one or two iOS samples in the pipeline.

Yes! under flutter/samples/experimental there is the date_planner app. This will continue to be iterated on. Next month I expect work will start on the second one, also in flutter/samples/experimental.

[sethladd]

Thanks all! To keep this issue focused on News Feed, sounds like that News Feed is no longer maintained.

If that's true, we should remove it from all docs (for example, see above) and clearly update the README for the News Feed app to mark it as Archived.

I'll see if there are any other feedback from folks before resolving this. When we resolve this issue, we'll open the sub-task issues.

[jmagman]

@hellohuanlin did work to make put-flutter-to-work idiomatic with SwiftUI https://github.com/flutter/put-flutter-to-work/pull/101, and I think it's our only add-to-app SwiftUI example. Updating the add-to-app sample app https://github.com/flutter/flutter/issues/150944 should be where we invest time, and redirect website links to that instead.

[sethladd]

Thanks for the update, @jmagman !

Sounds like I'm hearing:

1. Today, as of Aug 2024, the News Feed app is our current sample/example of SwiftUI + add2app + idiomatic updated code usage

2. We want to update our formal samples in flutter/samples to update their usage of idiomatic SwiftUI add2app code

3. Once we do (2) , we can formally archive News Feed as a project and remove News Feed from our docs (in preference for our updates samples)

Let me know if I missed something.

[hellohuanlin]

@hellohuanlin did work to make put-flutter-to-work idiomatic with SwiftUI flutter/put-flutter-to-work#101, and I think it's our only add-to-app SwiftUI example. Updating the add-to-app sample app flutter/flutter#150944 should be where we invest time, and redirect website links to that instead.

@sethladd Oh I misremembered - I mistakenly said yesterday we had an outdated SwiftUI example in flutter/examples repo, but actually we don't have any SwiftUI examples there.

[ericwindmill]

I'm closing this as a duplicate of #2402 because the end-goal of both is to have a SwiftUI add-to-app sample