ibis-bigquery
ibis-bigquery copied to clipboard
[BigQuery] `test_array_concat` is failing
Failing test
https://github.com/ibis-project/ibis/blob/a70d443c7931cb8bb47c52f97999589566e03cb2/ibis/tests/all/test_array.py#L11-L15
Test output
$ pytest ibis/tests/all/test_array.py::test_array_concat[BigQuery]
=============================== test session starts ===============================
platform darwin -- Python 3.7.8, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /Users/swast/src/ibis, inifile: setup.cfg
plugins: forked-1.2.0, mock-3.1.1, cov-2.10.0, xdist-1.34.0
collected 1 item
ibis/tests/all/test_array.py F [100%]
==================================== FAILURES =====================================
___________________________ test_array_concat[BigQuery] ___________________________
backend = <ibis.tests.backends.BigQuery object at 0x7fc3cbea21d0>
con = <ibis.bigquery.client.BigQueryClient object at 0x7fc3cbea2850>
@pytest.mark.xfail_unsupported
@pytest.mark.skip_missing_feature(
['supports_arrays', 'supports_arrays_outside_of_select']
)
# Issues #
#@pytest.mark.xfail_backends([BigQuery])
def test_array_concat(backend, con):
left = ibis.literal([1, 2, 3])
right = ibis.literal([2, 1])
expr = left + right
result = con.execute(expr)
> assert result == [1, 2, 3, 2, 1]
E ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
ibis/tests/all/test_array.py:18: ValueError
============================= short test summary info =============================
FAILED ibis/tests/all/test_array.py::test_array_concat[BigQuery] - ValueError: T...
================================ 1 failed in 1.33s ================================
Thoughts on fix
I suspect the query is returning correct results, but arrays are now coming back as numpy arrays. This is probably because google-cloud-bigquery is now using Arrow as an intermediate format before converting to a DataFrame.
Possible fixes:
- If a list object is required, add some additional conversion logic to convert arrays to lists.
- If a numpy object is okay, update the test to use a more general elementwise comparison.
Found in ibis-project/ibis#2353