ibis
ibis copied to clipboard
test: add streaming-specific test suite
Is your feature request related to a problem?
For the Flink and RisingWave backends, we run the generic test suite (ibis/backends/tests/) in batch mode and write customized tests for streaming ops, e.g. ibis/backends/flink/tests/. This will be a pain as we add more streaming backends.
Streaming-specific test suite will help
- @deepyaman: With identifying the common set of streaming functionalities different backends should support. I.e. instead of testing watermarks for each backend independently, should have a common, e.g.,
ibis/backends/tests/test_watermark.pythat will apply to all streaming backends. - @deepyaman: Ensure that the way things are implemented across streaming backends are the same (ie. the syntax for defining watermark can be copy-pasted between
RisingWave,Spark, andFlinkbackend, and new ones will make sure to follow the same).
Notes:
- @chloeh13q: Just in the three streaming backends that we have come across, there is already quite a bit of variation across them, in terms of syntax, vocabulary, etc. Instead of watermarks and sources and sinks and other DML/DDL things that tend to bear more variation, testing functionalities revolving around operations (e.g. windowing aggregations, asof joins) may be more interesting.
- @chloeh13q: Validating which ones of the TPC-H benchmarks are valid streaming queries might be needed because we have never actually explicitly tested these in streaming mode. Although it's also worth noting that Flink SQL and Spark SQL are both meant to have the same syntax across batch and streaming modes, apart from the fact that a lot of the times streaming mode supports a subset of batch mode and doesn't recognize specific syntax. An example of the top of my head is how Flink SQL in streaming mode only recognizes
CURRENT ROWandCAST(0 AS INTERVAL MINUTE)errors out. So we can possibly generalize by saying that if a SQL query works in streaming mode, it should work in batch mode, but it's not true the other way around.
Describe the solution you'd like
Need to investigate:
- How should a streaming-specific test suite look like?
- How much effort will this require?
What version of ibis are you running?
8.0.0
What backend(s) are you using, if any?
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct