python-bigquery-dataframes icon indicating copy to clipboard operation
python-bigquery-dataframes copied to clipboard

Optionally relax sqlglot / ibis pin via env var for use in SQLMesh

Open z3z1ma opened this issue 5 months ago • 1 comments

https://github.com/googleapis/python-bigquery-dataframes/blob/04139644af0c30c10519397fb3be7bebf6ec813c/setup.py#L63

There is very strong synergy between bigframes and https://github.com/TobikoData/sqlmesh However because of the sqlglot pin and the transitive sqlglot pin in the ibis pin, this library is incompatible with sqlmesh. (sqlmesh is built and maintained by creators of sqlglot and therefore usually has latest pin, and a strict one at that)

Since you are using setup.py, we should be able to use an env var or something similar to have an opt-in way to remove the version constraints on those two deps such that they can be defined by the consumer. We would accept that it opens one up to undefined behavior but by contrast it frees you from needing to constantly update your pins and allows more experimentation and adoption in the community.

You can see an example sqlmesh model using bigframes here https://github.com/TobikoData/sqlmesh/pull/2667/files

z3z1ma avatar Aug 30 '24 15:08 z3z1ma