databricks-sdk-py icon indicating copy to clipboard operation
databricks-sdk-py copied to clipboard

[ISSUE] workspace import fails if simplejson is not installed

Open jdavidheiser opened this issue 1 year ago • 0 comments

Description The Databricks API requires base64 encoded notebook content when using the import_ method to import files to the Workspace. The standard library json package fails when running json.dumps on bytestrings. The requests package uses simplejson if it is installed, and simplejson can handle bytestrings.

Reproduction Do the following in an environment without simplejson installed:

c = WorkspaceClient()
c.workspace._import(path=<your_path>, content=base64.b64encode("content".encode()))

Expected behavior The databricks SDK should probably include simplejson as a requirement, or the API should accept unicode encoded data.

Is it a regression? No, this has always been broken.

Debug Logs

in copy_file_to_databricks_workspace
    client.workspace.import_(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/service/workspace.py", line 2090, in import_
    self._api.do('POST', '/api/2.0/workspace/import', body=body, headers=headers)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/core.py", line 1147, in do
    return retryable(self._perform)(method,
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/retries.py", line 50, in wrapper
    raise err
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/retries.py", line 29, in wrapper
    return func(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/core.py", line 1225, in _perform
    response = self._session.request(method,
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/models.py", line 371, in prepare
    self.prepare_body(data, files, json)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/models.py", line 511, in prepare_body
    body = complexjson.dumps(json, allow_nan=False)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line [19], in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line [25]in iterencode
    return _iterencode(o, 0)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

jdavidheiser avatar Jan 17 '24 22:01 jdavidheiser