View test state in Street Explorer

[BudgieInWA]

When working on any code covered by the tests, you'll have a rapidly changing test source directory. Sometimes .orig files will show up that should be compared to new output.

It would be easy for the test browser to interact with an HTTP API to list and load the tests on the fly. A simple poll could detect .orig files, and offer to view the diff for those tests. (We could later progressively enhance to use a websocket to talk to a test runner that's watching the files.)

Note that we are moving away from yew in rust, and towards native JS.

Possibilities

trunk currently works as a perfectly cromulent web server with zero additional deps on the user.

trunk has a proxy setting that would let us easily pass a route through to whatever server we set up.

Serving files and directory indexes would get us 80% of the way there.

If we could write custom request handlers in rust, then we could do whatever fancyness we would need. We could even accept .osm (or a region to fetch) and create and run a new test.

Triggering tests that are run by the server would be cool, because the server can be set up to test in any way that we want, without needing to support the browser. E.g. we could generate a MapMachine rendering as part of the test or test any downstream users of osm2streets.

[BudgieInWA]

Alternatively: tests writes a summary at the root

When the tests are run, we could write everything we need into a nice tests.json at the root, that the client can load, parse, re-poll, etc.

Then the web app can be a purely static site. We could just move the site stuff into tests: index.html at the top; test-cases/ for test data; src/ for rust;web/ for js, css, assets. Serve with python -m http, nginx, trunk or whatever you have installed.

[BudgieInWA]

We now have street-explorer which is a static site that loads tests from the test directory. StreetExplorer should load more of the test data: it should show the description currently found in test.json and it should show both the .orig and "dirty" geometry so we don't need to run two versions to diff the geometry.

Having the test runner summarise the test run and write it to a file that StreetExplorer can display still sounds worthwhile to me.

I still think the tests themselves should come out of the src/ directory (they're not compiled source, they're runtime data).

[dabreegster]

it should show both the .orig and "dirty" geometry so we don't need to run two versions to diff the geometry.

I often generate details and flip between two tabs to spot diffs. The lane polygons and markings visually help. This'd be harder to do just displaying two files. But changing the viewport of the two browser tabs in a synchronized way is annoying, so at least having old and new in the same map could be useful.

Having the test runner summarise the test run and write it to a file that StreetExplorer can display still sounds worthwhile to me.

A simple answer could be to just have serve_locally.sh and the github action ls tests/src and write a file that we serve over HTTP. We could make a Rust method in the tests crate bake in the list at build-time using build.rs, but then we'd need to depend on a new crate from street-explorer and wire up WASM glue for it, which feels more convoluted

I still think the tests themselves should come out of the src/ directory (they're not compiled source, they're runtime data).

Sure, I don't recall why they're in src/ to begin with... oops?

[BudgieInWA]

I often generate details and flip between two tabs to spot diffs. The lane polygons and markings visually help. This'd be harder to do just displaying two files.

If StreetExplorer automatically displayed the diff the a local test-run generated (displaying both the orig and new at once, providing an button to toggle between) then it would be very appealing to use it I think. Then I'd just be tempted to add lane markings as another test output.

Having the test runner summarise the test run and write it to a file that StreetExplorer can display still sounds worthwhile to me.

A simple answer could be to just have serve_locally.sh and the github action ls tests/src and write a file that we serve over HTTP.

Yeah, I think that's what I'm thinking of... I was only thinking about the use case serving locally. If running the tests locally generated a test_results.json that lists all the failing tests, then StreetExplorer can load that in the same way it currently loads the test files. The local workflow would be

1. Run the tests, see that some failed. Currently it prints out "check the test diffs on geojson.io" or something, but that should link to localhost:8112/?test=blah
2. Run StreetExplorer (or have it already running)
3. Open StreetExplorer and browse the test diffs. StreetExplorer is just acting as a static client, loading the test data on demand.

[dabreegster]

Came up again today -- we could generate and serve a manifest of clean and dirty tests, to make it easier to inspect changes interactively