SpacetimeDB
SpacetimeDB copied to clipboard
Test coverage - SDK test suites
Rust SDK test suite is complete and merged (though will likely be extended in the future in response to new features or discovering bugs). We now need to port the tests to the C#, Python and TypeScript SDKs.
Original design doc:
- write test suites for all the different SDKs
- have minimum checklist for things that should be tested
- shared modules for test suites across all SDKs
- potentially comment out parts of them or have some other “expected failure” infra to support differences in SDK functionality
- e.g. C# does not support enums with payloads, so will need to avoid that test
- potentially comment out parts of them or have some other “expected failure” infra to support differences in SDK functionality
test flow
- create project with module and client
- deploy module
- run generate
- build sdk
- run test
to test
- subscribing, resubscribing
- calling reducer and getting response
- failing reducer and getting notified
- auth flow, both getting new tokens and re-auth with existing tokens
- express all (many?) types and ~generate~ bindings for them
- contexts
- table fields
- reducer arguments
- non-table type definitions
- types
- strings
- incl. strings that need escapes
- arrays
- of all the other types
- enums
- without payloads
- with payloads
- structs
- containing primitive types
- containing enums
- containing arrays
- bools
- integers
- floats
- maps?
- maybe left as TODO
- strings
- contexts
write test harness in rust
- run spacetimedb in-process
- see https://github.com/clockworklabs/SpacetimeDB/blob/master/crates/testing/tests/standalone_integration_test.rs
- run clients as subprocesses
- use their exit codes to communicate failure
- they print to stderr to describe failure
- offer
--describe <test>flag- prints to stdout what the test does
- then exits
- be as concurrent as possible
- this means running multiple test modules on the same stdb instance