elementary-data
Fill query_id for BigQuery
Is your feature request related to a problem? Please describe.
run_results table provides query_id but is empty on BigQuery.
BigQuery has a job id that could fit but is a bit special as it contains:
- a (GCP) project
- a location (or default to US)
- job_id (the actual "query id")
Describe the solution you'd like
As we can't have 3 fields, we could use Google's formatting for job id so that it looks like:
<gcp_project>:<location>.<job_id>
Then it can be reparsed easily and fit Google's "standard".
Describe alternatives you've considered
Using INFORMATION_SCHEMA.JOBS, we can get query field and extract the model and try to match run time to get information.
Additional context getting the query_id would allow to get job details such as query plan or other metadata that are worthwhile for further analysis. The raw data is available in the adapter in the adapter response.
Would you be willing to contribute this feature? I contributed to dbt-bigquery to add those fields so maybe I could look into it to add them in Elementary
