databricks-cli icon indicating copy to clipboard operation
databricks-cli copied to clipboard

Published 20 hours ago • verified publisher icon databricks

`databricks jobs configure --version=2.1` does not work

Open justinTM opened this issue 3 years ago • 7 comments

Steps:

# databricks runs list
WARN: Your CLI is configured to use Jobs API 2.0. In order to use the latest Jobs features please upgrade to 2.1: 'databricks jobs configure --version=2.1'. Future versions of this CLI will default to the new Jobs API. Learn more at https://docs.databricks.com/dev-tools/cli/jobs-cli.html
# databricks jobs configure --version=2.1
# databricks runs list
WARN: Your CLI is configured to use Jobs API 2.0. In order to use the latest Jobs features please upgrade to 2.1: 'databricks jobs configure --version=2.1'. Future versions of this CLI will default to the new Jobs API. Learn more at https://docs.databricks.com/dev-tools/cli/jobs-cli.html

justinTM avatar May 11 '22 21:05 justinTM

Is this bug still valid? We are getting the same warnings even though --version=2.1 is configured for jobs.

radoslawcz avatar Oct 20 '22 10:10 radoslawcz

How do you authenticate the CLI?

The command is supposed to write a key jobs-api-version at the right profile in your ~/.databrickscfg.

If you authenticate with environment variables it may persist but not pick up the value on the second call.

pietern avatar Oct 24 '22 14:10 pietern

If you are using #bash in cli then check your profile, use cat ~/.databrickscfg to see if your jobs api is using 2.0 or 2.1 TO change edit the profile ~/.databrickscfg using vi ~/.databrickscfg and change the Jobs API from 2.0 to 2.1, save the file :x then run your Jobs API 2.1 commands.

nskselva avatar Jan 04 '23 18:01 nskselva

If we authenticate with environment variables how we could set or use Jobs API 2.1?

jaimemeza avatar Jan 19 '23 16:01 jaimemeza

If we authenticate with environment variables how we could set or use Jobs API 2.1?

I'm also interested in hearing solution for this question? Currently this doesn't work.

skuisma avatar Feb 09 '23 08:02 skuisma

My solution was no use DATABRICKS_HOST and DATABRICKS_TOKEN, instead, I use DBX_HOST and DBX_TOKEN for example and manually create Databricks configuration file ~/.databrickscfg. That solves the issue at least for now.

jaimemeza avatar Feb 14 '23 20:02 jaimemeza

Apparently it should work with DATABRICKS_JOBS_API_VERSION -> https://github.com/databrickslabs/dbx/blob/57a70b33c144f0aa32f1321bfe964bd2c9470246/dbx/api/auth.py#L67

mehowthe avatar Feb 27 '23 15:02 mehowthe