databricks-sdk-go

databricks-sdk-go copied to clipboard

Description

Calling databricks bundles init returns the following error:

Error: failed to compute file content for {{.project_name}}/databricks.yml.tmpl. template: :35:31: executing "" at <user_name>: error calling user_name: unexpected error handling request: json: cannot unmarshal number into Go struct field APIErrorBody.error_code of type string. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues.

Reproduction Running databricks bundles init:

databricks bundle init    

Welcome to the default Python template for Databricks Asset Bundles!
Please provide the following details to tailor the template to your preferences.

Unique name for this project [my_project]: 
Include a stub (sample) notebook in 'my_project/src': yes
Include a stub (sample) Delta Live Tables pipeline in 'my_project/src': no
Include a stub (sample) Python package in 'my_project/src': no

Expected behavior Creation of databricks asset bundle

Is it a regression? using databricks cli v0.212.3

Debug Logs Request log:

GET /api/2.0/preview/scim/v2/Me
> * Host: 
> * Accept: application/json
> * Authorization: REDACTED
> * User-Agent: cli/0.212.3 databricks-sdk-go/0.30.1 go/1.21.6 os/darwin cmd/bundle_init auth/pat
< HTTP/2.0 403 Forbidden
< * Content-Length: 52
< * Content-Type: application/json; charset=utf-8
< * Date: Thu, 01 Feb 2024 07:11:53 GMT
< * Server: databricks
< * Vary: Accept-Encoding
< * X-Databricks-Reason-Phrase: Invalid access token.
< {
<   "error_code": 403,
<   "message": "Invalid access token."
< }

Other Information

• OS: macOS
• Version: 14.3

Additional context Add any other context about the problem here.

Having the same issue.

Hey @mzwiesler , I solved it for me. Do you have multiple user profiles in your .databrickscfg file?

Hi @christopher-danz Yes I have a couple of them and a default one. Tried the command with profile and without but nothing worked so far. Did you solve it?

I feel like I tried everything there is to troubleshoot.. empty cache, create new .databrickscfg file, generate new token, ... the whole thing and I am not entirely sure what made the differnce in the end..

After ensuring you authenticate correctly, try to run: databricks jobs list ... it should list all the jobs for your DEFAULT profile.

then run: databricks jobs list --profile=YOUR_DESIRED_PROFILE_TO_USE_DAB and it should list the jobs for the other host, right?

I then ran: databricks bundle init (specified the unique name and the additional stuff) and it worked then.

I hope this will solve your issue, if not let me know. Maybe I can further assist you.

Hi @christopher-danz True it seems it really was a token issue. It now works with a new token. Still not 100% sure what happened. Thanks for your support.

I'm glad it worked out for both of us. 😊

Similar issue here, I'm using an asset bundle generated with databricks bundle init mlops-stacks, the CI/CD is set to run databricks bundle validate with a DATABRICKS_TOKEN env var, but it's failing with that "cannot unmarshal" error

Error: unexpected error handling request: json: cannot unmarshal number into Go struct field APIErrorBody.error_code of type string. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:

GET /api/2.0/preview/scim/v2/Me
> * Host: 
> * Accept: application/json
> * Authorization: REDACTED
> * Traceparent: 00-fdc3a9a[7](https://github.com/KardFinancial/merchant-identification/actions/runs/9898272483/job/27344733076#step:4:8)5520e3039777b7e44a23e6[8](https://github.com/KardFinancial/merchant-identification/actions/runs/9898272483/job/27344733076#step:4:9)f-8af3b60ecbd5fbc2-01
> * User-Agent: cli/0.221.0 databricks-sdk-go/0.42.0 go/1.21.10 os/linux cmd/bundle_validate auth/pat cicd/github
< HTTP/2.0 401 Unauthorized
< * Content-Length: [9](https://github.com/KardFinancial/merchant-identification/actions/runs/9898272483/job/27344733076#step:4:10)8
< * Content-Type: application/json; charset=utf-8
< * Date: Thu, [11](https://github.com/KardFinancial/merchant-identification/actions/runs/9898272483/job/27344733076#step:4:12) Jul 2024 20:48:14 GMT
< * Server: databricks
< * Vary: Accept-Encoding
< * Www-Authenticate: ***"DatabricksRealm"
< * X-Databricks-Reason-Phrase: Credential was not sent or was of an unsupported type for this API.
< {
<   "error_code": 401,
<   "message": "Credential was not sent or was of an unsupported type for this API."
< }

I am running into the same issue as joe using a token to deploy DABs.

I've updated the databricks CLI version my DAB's using from v0.221.0 to v0.229.0, since this issue mentioned a fix. Databricks CLI v0.229.0 uses Go SDK 0.47.0, but the github action is still failing to run databricks bundle validate, now with a more vague message Process completed with exit code 1.

databricks bundle validate -t stage > ../validate_output.txt
  shell: /usr/bin/bash -e {0}
  env:
    STAGING_WORKSPACE_TOKEN: ***
    PROD_WORKSPACE_TOKEN: ***
    DATABRICKS_TOKEN: ***
Error: Process completed with exit code 1.

Also, using databricks/[email protected] had the error

  shell: /usr/bin/bash -e {0}
  env:
    DATABRICKS_TOKEN: ***
Error: Credential was not sent or was of an unsupported type for this API.

What worked for me was, instead of setting a DATABRICKS_TOKEN, setting a DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET generated for the service principal. E.g.

env:
  DATABRICKS_TOKEN: ${{ secrets.STAGING_WORKSPACE_TOKEN }}

becomes

env:
  DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }} 
  DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}

Here's how I generated the secret, you do have to be an admin.

Used databricks/[email protected].