gcp-bigquery-client
gcp-bigquery-client copied to clipboard
JobApi query() method only returns partial results
It is really unclear from the comments/documentation that this method leaves off some data if query result is big enough
Check this document where they mention about this. only 10 seconds until the query runs and then it sends results. https://docs.rs/gcp-bigquery-client/latest/gcp_bigquery_client/model/query_request/struct.QueryRequest.html#structfield.timeout_ms
I do have the same problem, changing timeout_ms
does not help (also, the number of returned rows is the same in all reruns... so it seems to be some other flag, rather than a timeout)
also, max_results
has no effect (if increased)
fwiw, I did move to a JobConfigurationQuery
approach, just like in the pagination example, and that works now (setting a sensible, small pagination size)