Include job_id, run_id and task_id in adapter_response in the results on-run-end context variable

[tomsky]

trafficstars

Describe the feature

Make the job_id, run_id and task_run_id from job runs in Databricks available in the adapter_response in run_results.json. Currently I only see query_id and _message available in the adapter_response. I would like to be able to connect a dbt run to a Databricks job run from system.workflow.job_run_timeline.

Describe alternatives you've considered

Passing the job_id and run_id as vars to dbt with vars command might work, but this would require the vars added to all run commands.

dbt run --vars '{"job_id": "{{job_id}}"}'

Passing the job_id as env var to the cluster was unsuccessful.