ibis icon indicating copy to clipboard operation
ibis copied to clipboard

bug: BigQuery Create table with partition_by argument is not working

Open ruiyang2015 opened this issue 5 months ago • 4 comments

What happened?

I am trying to use the BigQuery backend to create a table with a partition by expression, ibis is raising exception. here is a simple code to reproduce the error:

import ibis
c = ibis.bigquery.connect(...)
schema = ibis.schema([ ("c1", "date"), ("c2", "int")])
c.create_table('test_tb', partition_by='DATE_TRUNC(c1, MONTH)') # -> this line is throwing exception 

What version of ibis are you using?

9.4.0

What backend(s) are you using, if any?

BigQuery

Relevant log output

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/ibis/backends/bigquery/__init__.py", line 1007, in create_table
    raise com.IbisError("One of the `schema` or `obj` parameter is required")
ibis.common.exceptions.IbisError: One of the `schema` or `obj` parameter is required
>>> c.create_table('test_tb', partition_by='DATE_TRUNC(c1, MONTH)', schema=schema)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/ibis/backends/bigquery/__init__.py", line 1099, in create_table
    self.raw_sql(sql)
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/ibis/backends/bigquery/__init__.py", line 710, in raw_sql
    return self.client.query_and_wait(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/cloud/bigquery/client.py", line 3601, in query_and_wait
    return _job_helpers.query_and_wait(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/cloud/bigquery/_job_helpers.py", line 414, in query_and_wait
    return _wait_or_cancel(
           ^^^^^^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/cloud/bigquery/_job_helpers.py", line 562, in _wait_or_cancel
    return job.result(
           ^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/cloud/bigquery/job/query.py", line 1676, in result
    while not is_job_done():
              ^^^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
           ^^^^^^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
             ^^^^^^^^
  File "/Users/ruiyang/Go/src/github.com/ascend-io/ascend-core/.venv/lib/python3.11/site-packages/google/cloud/bigquery/job/query.py", line 1625, in is_job_done
    raise job_failed_exception
google.api_core.exceptions.BadRequest: 400 Unrecognized name: D at [1:111]; reason: invalidQuery, location: query, message: Unrecognized name: D at [1:111]


### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

ruiyang2015 avatar Sep 04 '24 22:09 ruiyang2015