egui icon indicating copy to clipboard operation
egui copied to clipboard

Add egui testing library

Open lucasmerlin opened this issue 1 year ago • 3 comments

  • closes #3491
  • closes #3926

This adds a testing library to egui based on kittest. Kittest is a new AccessKit-based testing library. The api is inspired by the js testing-library where the idea is also to query the dom based on accessibility attributes. We made kittest with egui in mind but it should work with any rust gui framework with AccessKit support.

It currently has support for:

  • running the egui app, frame by frame
  • building the AccessKit tree
  • ergonomic queries via kittest
    • via e.g. get_by_name, get_by_role
  • simulating events based on the accesskit node id
  • creating arbitrary events based on Harness::input_mut
  • rendering screenshots via wgpu
  • snapshot tests with these screenshots

A simple test looks like this:

fn main() {
    let mut checked = false;
    let app = |ctx: &Context| {
        CentralPanel::default().show(ctx, |ui| {
            ui.checkbox(&mut checked, "Check me!");
        });
    };

    let mut harness = Harness::builder().with_size(egui::Vec2::new(200.0, 100.0)).build(app);
    
    let checkbox = harness.get_by_name("Check me!");
    assert_eq!(checkbox.toggled(), Some(Toggled::False));
    checkbox.click();
    
    harness.run();

    let checkbox = harness.get_by_name("Check me!");
    assert_eq!(checkbox.toggled(), Some(Toggled::True));

    // You can even render the ui and do image snapshot tests
    #[cfg(all(feature = "wgpu", feature = "snapshot"))]
    egui_kittest::image_snapshot(&egui_kittest::wgpu::TestRenderer::new().render(&harness), "readme_example");
}

~Since getting wgpu to run in ci is a hassle, I'm taking another shot at creating a software renderer for egui (ideally without a huge dependency like skia)~ (this didn't work as well as I hoped and it turns out in CI you can just run tests on a mac runner which comes with a real GPU)

Here is a example of a failed snapshot test in ci, it will say which snapshot failed and upload an artifact with the before / after and diff images: https://github.com/emilk/egui/actions/runs/11183049487/job/31090724606?pr=5166

lucasmerlin avatar Sep 25 '24 10:09 lucasmerlin

Preview available at https://egui-pr-preview.github.io/pr/5166-lucas/testing_library Note that it might take a couple seconds for the update to show up after the preview_build workflow has completed.

github-actions[bot] avatar Sep 25 '24 10:09 github-actions[bot]

After some discussion we came to these conclusions:

  • Investigate using git lfs for storing the .png snapshot files so we don't bloat the git repository. Try to set it up in such a way that git clone + cargo test still works for contributors without lfs.
  • Try to set up a proper software wgpu backend that works on CI, same as the wgpu repository uses. This will be the most powerful and accurate, and allow us to test more than just egui (i.e. the Rerun Viewer)
  • Move the accesskit-specific part of the integration to its own repository and eventually invite collaborators from other GUI toolkits to collaborate on it

emilk avatar Sep 25 '24 15:09 emilk

I've changed everything you mentioned and also added Harness::wgpu_snapshot as a convenicence.

Also, I've disabled dithering since it caused minor differences between frames. This made the pngs much smaller!

lucasmerlin avatar Oct 12 '24 13:10 lucasmerlin