Tim Sweña (Swast) comments

Results 302 comments of


                                            Tim Sweña (Swast)

test `read_gbq` with `dryRun` in `configuration` parameter

Unfortunately even with googleapis/python-bigquery-pandas#152, still can't do dry run queries because it raises when google-cloud-bigquery tries to fetch the results. Test code (query from [analyzing PyPI downloads](https://packaging.python.org/guides/analyzing-pypi-package-downloads/#counting-package-downloads)): ```python def test_configuration_with_dryrun(self):...

test `read_gbq` with `dryRun` in `configuration` parameter

I've confirmed that `dryRun` queries do work upstream in https://github.com/GoogleCloudPlatform/google-cloud-python/pull/5119 . I think pandas-gbq will need to check for dry run queries and decide not to try to fetch the...

ENH: `DataFrame.struct.explode(column, *, separator=".")` method to pull struct subfields into the parent DataFrame

> From an API design perspective, having a dtype-specific accessor on DataFrame seems like a bad approach. Gotcha. I did see we have some already. e.g. [SparseFrameAccessor](https://github.com/pandas-dev/pandas/blob/6fa4eb43fbf01d558c9e8cd0fdde6fa5359c9d19/pandas/core/arrays/sparse/accessor.py#L242), which does seem...

chore: rotate ibis bot keys

I'm getting pinged internally about this for the BQ keys. Happy to help how I can. Perhaps we can use https://github.com/google-github-actions/auth to avoid needing key files for the BQ auth?

Support JSON data type

Hello @jlynchMicron , thanks for the feedback. We are actively working on JSON support, for example by adding some JSON functions to the `bigframes.bigquery` package (see: https://cloud.google.com/python/docs/reference/bigframes/latest/bigframes.bigquery#bigframes_bigquery_json_set). Based on the...

Error in trying to upload Dataframe with a an Arrow-backed list of large_strings

If you have > 2 GB of data in a single string, you won't be able to write it to BigQuery anyway. The row size limit is 100 MB.

Error in trying to upload Dataframe with a an Arrow-backed list of large_strings

Marking this as a feature request for `large_string` support.

feat: `to_json_string` (alternatively, `to_json`, but that could get confusing with the JSON type)

BigQuery doesn't support casting to JSON directly. ``` WITH Sequences AS ( SELECT [0, 1, 1, 2, 3, 5] AS some_numbers UNION ALL SELECT [2, 4, 8, 16, 32] UNION...

feat: add `first_value` and `last_value` metrics with and "ignore nulls" option for use in a window

> In SQLGlot, it does appear there is a [First](https://github.com/tobymao/sqlglot/blob/89fc63c5831dc5d63feff9e39fea1e90d65e9a09/sqlglot/expressions.py#L4917) expression with at least some [IGNORE NULLS support](https://github.com/tobymao/sqlglot/blob/89fc63c5831dc5d63feff9e39fea1e90d65e9a09/sqlglot/dialects/hive.py#L300). My work is in-progress to see if this operator is compatible with...

feat: add `first_value` and `last_value` metrics with and "ignore nulls" option for use in a window

Filed https://github.com/tobymao/sqlglot/issues/3749 as I am having trouble figuring out how to create the SQLGlot expression. Other examples I found in the compiler use `self.agg.avg(...)`, but I don't think that would...