on spec update, breakage in types used only by preview routes (and types unused by any routes) should be ignored for semver purposes

[wfraser]

Why is this feature valuable to you? Does it solve a problem you're having? On spec update, we run the tests for the current version against code generated for the new version, and if tests fail, we assume the update is semver-incompatible.

This is mostly correct, however there are "preview" routes which are not covered under our stability guarantee, and breakage to these routes shouldn't necessitate a semver-incompatible version bump. Also occasionally types for routes which have not yet been published will end up in the spec, and these "unreachable" types shouldn't be considered breakage either.

Describe the solution you'd like The difficulty lies in the fact that only routes are tagged as preview, but the tests mostly cover types. What we should do is build a dependency tree and extend the unstable feature to any types which are only referred to by preview routes. This will improve documentation as well. Then when doing the spec update build, we can disable the unstable feature and these tests will not be run.

Similarly, the dependency tree can find "unreachable" types, and their tests should be marked with #[ignore] since their correctness mostly doesn't matter: no routes use the types anyway.

Describe alternatives you've considered Currently I do this with manual review, which is tedious, but this edge case happens fairly infrequently, so the cost is fairly low.

Additional context For a real example, the 2022-10-11 spec update makes incompatible changes to the openid namespace, which only has preview routes.

Also prior to the 2022-10-01 spec update the openid namespace had unreachable types due to not having any routes.