planetary-computer-sdk-for-python icon indicating copy to clipboard operation
planetary-computer-sdk-for-python copied to clipboard

Assign `sas_token`, not `credential`, for adlfs

Open gadomski opened this issue 3 years ago • 1 comments
trafficstars

Solution was presented in https://github.com/fsspec/adlfs/issues/346#issuecomment-1239095846. Adds some dev dependencies to get a real unit test.

Without this fix, the following error occurs on weakref cleanup (planetary-computer version 0.4.7, adlfs version 2022.9.1):

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/weakref.py", line 667, in _exitfunc
    f()
  File "/usr/local/Cellar/[email protected]/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/weakref.py", line 591, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/Users/gadomski/.virtualenvs/planetary-computer-tasks/lib/python3.10/site-packages/fsspec/asyn.py", line 96, in sync
    raise return_result
  File "/Users/gadomski/.virtualenvs/planetary-computer-tasks/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
    result[0] = await coro
  File "/Users/gadomski/.virtualenvs/planetary-computer-tasks/lib/python3.10/site-packages/adlfs/utils.py", line 75, in close_credential
    await file_obj.credential.close()
AttributeError: 'str' object has no attribute 'close'

I real-world tested this w/ adlfs access of a geoparquet table -- I haven't tried it with the other data types.

gadomski avatar Sep 21 '22 19:09 gadomski

Thanks for this. I'm going to look at the linked issue a bit.

Edit: see https://github.com/fsspec/adlfs/issues/346#issuecomment-1254150524. I'm hoping how we're using adlfs is OK.

TomAugspurger avatar Sep 21 '22 19:09 TomAugspurger

This was handled upstream in https://github.com/fsspec/adlfs/pull/351. Thanks anyway.

TomAugspurger avatar Nov 03 '22 13:11 TomAugspurger