RFC: Allow application to build its own AccessKit subtree under a Ui

[delan]

• Closes https://github.com/emilk/egui/issues/THE_RELEVANT_ISSUE
• [ ] I have followed the instructions in the PR template

[delan]

in servoshell, we need to build AccessKit nodes for web content in the webview, and combine it with the AccessKit tree for the egui widgets in the surrounding UI. it seems egui does not yet provide a way for applications to build arbitrary subtrees, but we think we can do it with something like Context::accesskit_node_builder() except the caller can also modify the AccessKitPassState:

ctx.accesskit_subtree_builder(ui.id(), |node, accesskit_state| {
    // configure the node for this Ui
    node.set_role(Role::Group);

    // create and configure a child node.
    // unlike the node above, we need to insert it into the AccessKitPassState.
    let child_id = ui.id().with(1);
    let mut child = Node::default();
    child.set_role(Role::Switch);
    accesskit_state.nodes.insert(child_id, child);

    // attach the child to the node for this Ui.
    node.push_child(child_id.value().into());
});

does this approach seem reasonable?

[github-actions[bot]]

Preview available at https://egui-pr-preview.github.io/pr/7679-accesskit-subtree-builder Note that it might take a couple seconds for the update to show up after the preview_build workflow has completed.

View snapshot changes at kitdiff

[delan]

so we went away and played with that, and found that while building our custom subtree in accesskit_subtree_builder() was useful, it forced us to rebuild that subtree on every pass. we were able to solve that by sending our own accesskit tree updates independently of egui, although this required minor changes to egui-winit (45b72c1b592ab0de498d8315676d0ecb8504170a):

struct MyApp {
    root_accesskit_node_id: Option<egui::accesskit::NodeId>,
    accesskit_adapter: Arc<RwLock<accesskit_winit::Adapter>>,
}

// on every pass, inside context.run…
ctx.accesskit_subtree_builder(ui.id(), |node, accesskit_state| {
    node.set_role(Role::Group);

    // egui sends a TreeUpdate on every pass with all of the nodes it knows about.
    // but TreeUpdate can be used incrementally, so we can take advantage of that
    // to send updates to Servo’s accessibility subtree on our own schedule.
    let root_accesskit_node_id = my_app.root_accesskit_node_id.get_or_insert_with(|| {
        // the first time only, we tell accesskit about the root of our tree using
        // a dummy node, which we can later update however we like.
        let child = Node::default();
        let child_id = ui.id().with(1);
        accesskit_state.nodes.insert(child_id, child);
        child_id.value().into()
    });
    // to ensure that the boundary between egui’s tree and our tree doesn’t get
    // clobbered, we need to tell egui to include the root of our tree at the node
    // where they meet. then we can do what we want with that root.
    node.push_child(*root_accesskit_node_id);
});

// later, at any time (outside of context.run)...
if let Some(root_id) = my_app.root_accesskit_node_id {
    let mut accesskit_adapter = my_app.accesskit_adapter.write().unwrap();
    accesskit_adapter.update_if_active(|| {
        // create a subtree rooted at the node with id `root_accesskit_node_id`,
        // which is the same as the id of the dummy node we created in
        // `ctx.accesskit_subtree_builder()`.
        let mut root = Node::default();
        root.set_role(Role::WebView);
        // TODO: we’re still working on a way to generate unique accesskit ids
        let a_id = generate_unique_id();
        let mut a = Node::default();
        a.set_role(Role::Button);
        let b_id = generate_unique_id();
        let mut b = Node::default();
        b.set_role(Role::Button);
        let c_id = generate_unique_id();
        let mut c = Node::default();
        c.set_role(Role::Button);
        root.set_children(vec![a_id, b_id, c_id]);
        // because we used that same id, accesskit will combine this subtree
        // with egui’s tree, and the two trees can update independently.
        TreeUpdate {
            nodes: vec![
                (root_id, root),
                (a_id, a),
                (b_id, b),
                (c_id, c),
            ],
            tree: None,
            // TODO: this needs to align with the focus in egui’s updates,
            // unless the focus has genuinely changed
            focus: b_id,
        }
    });
}

[lucasmerlin]

I wonder if you could do this via the new Plugin trait. It has a output_hook that lets you inspect / modify the PlatformOutput which also contains the accesskit output. Could you just push your accesskit nodes via that?