gcp-bigquery-client icon indicating copy to clipboard operation
gcp-bigquery-client copied to clipboard

JobApi query() method only returns partial results

Open adensur opened this issue 2 years ago • 3 comments

It is really unclear from the comments/documentation that this method leaves off some data if query result is big enough

adensur avatar Jul 19 '22 14:07 adensur

Check this document where they mention about this. only 10 seconds until the query runs and then it sends results. https://docs.rs/gcp-bigquery-client/latest/gcp_bigquery_client/model/query_request/struct.QueryRequest.html#structfield.timeout_ms

rahul-soshte avatar Apr 03 '23 11:04 rahul-soshte

I do have the same problem, changing timeout_ms does not help (also, the number of returned rows is the same in all reruns... so it seems to be some other flag, rather than a timeout)

also, max_results has no effect (if increased)

chroth7 avatar Jan 10 '24 14:01 chroth7

fwiw, I did move to a JobConfigurationQuery approach, just like in the pagination example, and that works now (setting a sensible, small pagination size)

chroth7 avatar Jan 10 '24 15:01 chroth7