databricks-sdk-py
databricks-sdk-py copied to clipboard
[ISSUE] workspace import fails if simplejson is not installed
Description
The Databricks API requires base64 encoded notebook content when using the import_
method to import files to the Workspace. The standard library json
package fails when running json.dumps
on bytestrings. The requests
package uses simplejson
if it is installed, and simplejson
can handle bytestrings.
Reproduction Do the following in an environment without simplejson installed:
c = WorkspaceClient()
c.workspace._import(path=<your_path>, content=base64.b64encode("content".encode()))
Expected behavior The databricks SDK should probably include simplejson as a requirement, or the API should accept unicode encoded data.
Is it a regression? No, this has always been broken.
Debug Logs
in copy_file_to_databricks_workspace
client.workspace.import_(
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/service/workspace.py", line 2090, in import_
self._api.do('POST', '/api/2.0/workspace/import', body=body, headers=headers)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/core.py", line 1147, in do
return retryable(self._perform)(method,
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/retries.py", line 50, in wrapper
raise err
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/retries.py", line 29, in wrapper
return func(*args, **kwargs)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/databricks/sdk/core.py", line 1225, in _perform
response = self._session.request(method,
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/sessions.py", line 575, in request
prep = self.prepare_request(req)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/sessions.py", line 486, in prepare_request
p.prepare(
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/models.py", line 371, in prepare
self.prepare_body(data, files, json)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/requests/models.py", line 511, in prepare_body
body = complexjson.dumps(json, allow_nan=False)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line [19], in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line [25]in iterencode
return _iterencode(o, 0)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable