SpacetimeDB icon indicating copy to clipboard operation
SpacetimeDB copied to clipboard

Add snapshot tests for normalized ModuleDef

Open RReverser opened this issue 1 year ago • 3 comments

Description of Changes

This provides snapshot testing for ModuleDef returned from Wasm modules. It will automatically compare ModuleDefs of test modules against both the previous snapshots, and across different languages.

Note that comparison doesn't (can't) use raw ModuleDef. There are some minor non-functional differences between Rust and C# that need to be cleaned up before comparing - most notably, the order in which types are registered during table traverse, plus some smaller things like Rust registering column constraints even where they're no-op. I believe those changes don't matter and are fine to remove before comparing.

The history is split into two commits - one that adds working testing with all the cleanups, and in a separate one I added more compact debug representations for AlgebraicType and child types. That commit could be reverted, but it makes the snapshots much easier to read (see the diff) and I believe will be useful for regular debugging outside this test as well.

Fixes #1589.

API and ABI breaking changes

If this is an API or ABI breaking change, please apply the corresponding GitHub label.

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected!

  • [x] Ran SDK tests (cargo test -p spacetimedb-sdk).
  • [ ] Write a test you want a reviewer to do here, so they can check it off when they're satisfied.

RReverser avatar Aug 20 '24 18:08 RReverser

This is great, but possibly should be done against the new ModuleDef type from #1572 and #1606? Those are going to merge this week, they have been a little delayed because I kept getting blocked on other PRs merging. I would really love to implement pretty-printing for that ModuleDef.

kazimuth avatar Aug 21 '24 17:08 kazimuth

That's fine by me; I don't think it will be hard to adapt tests either way.

RReverser avatar Aug 21 '24 17:08 RReverser

@kazimuth I rebased this, but on the 2nd thought we shouldn't compare V9 moduledef just yet.

The purpose of the original issue / of adding those tests was to verify that the module structure used for generation is the same across different implementations (Rust and C#) of the same module.

Since spacetime generate and extract_descriptions which we verify here still uses the V8 module definition, I think we should merge this as-is first and upgrade the test only after generation for V9 is implemented, otherwise we won't be comparing the same structures that generate uses.

Meanwhile, we'll at least already have some testing in place.

RReverser avatar Sep 04 '24 16:09 RReverser