ibis icon indicating copy to clipboard operation
ibis copied to clipboard

test: move backend-specific backend tests to the larger suite

Open anjakefala opened this issue 3 years ago • 4 comments

Issue rephrased from: https://github.com/ibis-project/ibis/pull/3988#issuecomment-1138599760.

Since https://github.com/ibis-project/ibis/pull/3988, database loading is done automatically by the pytest backend fixtures. In theory, ci/datamgr.py load should not be necessary anymore.

However, we cannot remove ci/datamgr.py load from the CI. Some of the backends have tests that run against backend data, but aren't "backend" tests in that they do not run across all the backends. The non-backend-backend tests and the proper backend tests expect the same loaded data. Without loading that data at some point, the tests will fail to find the things they need to run the test.

We want to move the backend-specific backend tests to the larger suite. We'll have to go test by test and remove duplicated tests as well as notimpl/notyet/never-mark the ones that don't work on the other backends.

The places to start looking are ibis/backends/{sqlite,postgres,clickhouse}/tests/test_functions.py. Much of the duplicated backend testing lies there.

anjakefala avatar May 26 '22 23:05 anjakefala

Example here: https://github.com/ibis-project/ibis/pull/4070

anjakefala avatar Jun 09 '22 16:06 anjakefala

/take

anjakefala avatar Jun 13 '22 17:06 anjakefala

I have some time right now to tackle this issue. Since https://github.com/ibis-project/ibis/pull/4087 ci/datamgr.py load is not needed anymore. Do we still want to move the backend-specific tests into the larger suite?

anjakefala avatar Jul 11 '22 16:07 anjakefala

I think we do. I suspect this will need to be tackled in a few PRs, since there are a good number of backend specific tests.

I'll suggest the following order:

  1. Move the SQL backend tests
  2. Look into moving the pandas tests if they are generalizable
  3. ^ same for the dask backend
  4. Delete the backend specific test directories with prejudice.

cpcloud avatar Jul 11 '22 17:07 cpcloud

Let's close this out! We can continue to do this as time permits.

Thanks for your work on this @anjakefala!

cpcloud avatar Nov 20 '22 14:11 cpcloud