[TEST PR PLZ IGNORE] sample test framework

[tyler-hoffman]

This is intended to show how we can instrument the tests for CORE-436. I would recommend starting by looking at the new tests themselves, rather than the conftest.py stuff.

About the tests

We want as little magic here as we can. Unfortunately, we were stuck with either having to create our own decorator, or use request.getfixturevalue, and the former seems more ergonomic. A few things to note in the test:

• batch_for_datasource is magically injected by parameterize_batch_for_data_sources. (sorry!)
• In some cases, we likely want to just let the tooling figure out the schema for our SQL types, but in other cases, we may want to be explicit on types (time vs datetime, etc), so we're allowing users to optionally specify the types, as shown in the second test
• Thinking about it now, I'm not sure we need the optional description param on batch_for_datasource.

About the "framework" or whatever

The interface is hopefully clear from the sample tests, but here are the high level details on the various concepts:

• BatchSetup: these classes are responsible for setting up and tearing down data. They return batches in this example, but we could just as easily return checkpoints, validation definitions, dataclasses holding all of them, etc. I think batches are the simplest thing if we really are after testing expectations against datasources, as opposed to testing that e.g. checkpoints work
• DataSourceConfig: these classes are the public interface to be used by tests. They should be minimal, and for now essentially do 3 things:
  • Determine the pytest mark
  • Determine the label to show up when the test is run
  • Instantiate the BatchSetup Ce could probably merge the 2 classes, or have a helper derive a BatchSetup from the DataSourceConfig - I don't have strong opinions here, but it seemed like a reasonable structure

• [ ] Description of PR changes above includes a link to an existing GitHub issue
• [ ] PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB]
• [ ] Code is linted - run invoke lint (uses ruff format + ruff check)
• [ ] Appropriate tests and docs have been updated

For more information about contributing, see Contribute.

After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!