ibis icon indicating copy to clipboard operation
ibis copied to clipboard

test(pyspark): verify the existing backend-specific tests work in streaming mode

Open mfatihaktas opened this issue 1 year ago • 0 comments

Is your feature request related to a problem?

As described in the meta-issue https://github.com/ibis-project/ibis/issues/8868, our initial understanding is that Spark SQL has the same syntax for both batch and streaming. This is why, anything at the intersection of these two should just work.

Describe the solution you'd like

We need to verify our understanding (stated above) by testing the existing pyspark backend functionality in streaming mode. The first part of this is to verify the pyspark-specific tests under ibis/backends/pyspark/tests/ in streaming mode.

What version of ibis are you running?

8.0.0

What backend(s) are you using, if any?

pyspark

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

mfatihaktas avatar Apr 04 '24 14:04 mfatihaktas