emilk
Add pixel_threshold and improve threshold docs
Sometimes visual threshold isn't enough and it would be nice to just allow x pixels to be different
Workaround we use right now:
let broken_percent_threshold = 0.003;
let num_pixels = (size.x * size.y).ceil() as f64;
use re_viewer_context::external::egui_kittest;
match harness.try_snapshot(name) {
Ok(_) => {}
Err(err) => match err {
egui_kittest::SnapshotError::Diff {
name,
diff: num_broken_pixels,
diff_path,
} => {
let broken_percent = num_broken_pixels as f64 / num_pixels;
re_log::debug!(num_pixels, num_broken_pixels, broken_percent);
assert!(
broken_percent <= broken_percent_threshold,
"{name} failed because {broken_percent} > {broken_percent_threshold}\n{diff_path:?}"
)
}
_ => panic!("{name} failed: {err}"),
},
}
@Wumpf Do we still need this after mergning
- https://github.com/rerun-io/rerun/pull/8971 ?
Yes absolutely. Even with AA disabled, we still have one (down from several) test where this is needed and there is many reasons why we may run into problems that need this.
The problem with the workaround we're using right now is that whenever we react to a egui_kittest::SnapshotError::Diff { and discard its error as passing, the image has already been updated. I.e. if I'm on a machine that didn't produce the original and run all tests with UPDATE_SNAPSHOTS=1, I end up having a git diff with this despite passing the test.
@lucasmerlin would you be up for incorporating this in?
