TileDB-Py
TileDB-Py copied to clipboard
TileDB-Py built without GCS Support
I encountered an error when trying to point to a TileDB array from Google Cloud storage. The catch is, it only happens if tiledb-py was installed a specific way. For instance:
After installing TileDB into my conda environment with conda install -c conda-forge tiledb tiledb-py
(TileDB version 2.10.3, tiledb-py version 0.16.3), then calling dask.array.from_tiledb('gcs://my-bucket/array.tldb')
after setting os.environ('GOOGLE_APPLICATION_CREDENTIALS')='/path/to/token.json'
, I receive the error:
---------------------------------------------------------------------------
TileDBError Traceback (most recent call last)
Input In [5], in <cell line: 3>()
1 os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = token
2 tic1 = time.time()
----> 3 da = dsa.from_tiledb(root + data + '.tldb',
4 storage_options={'sm.compute_concurrency_level': 1, 'sm.io_concurrency_level': 1})
5 toc1 = time.time()
6 connectTime = toc1 - tic1
File /contrib/Jacob.Green/miniconda3/envs/cloud-data/lib/python3.10/site-packages/dask/array/tiledb_io.py:48, in from_tiledb(uri, attribute, chunks, storage_options, **kwargs)
46 tdb = uri
47 else:
---> 48 tdb = tiledb.open(uri, attr=attribute, config=tiledb_config, key=key)
50 if tdb.schema.sparse:
51 raise ValueError("Sparse TileDB arrays are not supported")
File /contrib/Jacob.Green/miniconda3/envs/cloud-data/lib/python3.10/site-packages/tiledb/highlevel.py:23, in open(uri, mode, key, attr, config, timestamp, ctx)
9 def open(uri, mode="r", key=None, attr=None, config=None, timestamp=None, ctx=None):
10 """
11 Open a TileDB array at the given URI
12
(...)
21 :return: open TileDB {Sparse,Dense}Array object
22 """
---> 23 return tiledb.Array.load_typed(
24 uri,
25 mode=mode,
26 key=key,
27 timestamp=timestamp,
28 attr=attr,
29 ctx=_get_ctx(ctx, config),
30 )
File tiledb/libtiledb.pyx:3577, in tiledb.libtiledb.Array.load_typed()
File tiledb/libtiledb.pyx:3403, in tiledb.libtiledb.preload_array()
File tiledb/libtiledb.pyx:575, in tiledb.libtiledb._raise_ctx_err()
File tiledb/libtiledb.pyx:560, in tiledb.libtiledb._raise_tiledb_error()
TileDBError: [TileDB::VFS] Error: TileDB was built without GCS support
However, if I install TileDB with pip install tiledb
, everything works as intended.
Hi @jacobgreen405,
We apologize for the inconvenience. We are currently working on fixing GCS support for the tiledb
conda packages at https://github.com/conda-forge/tiledb-feedstock/pull/161.
As you mention, the current workaround is to use the PyPI wheels instead.
Thanks.
Hi all,
Is this issue solved? I see the same error with tiledb-py version 0.20.0.
I'm running on a Mac M2. Is it possible it's something related to tiledb-py for the arm64 architecture not being built with GCS support?
Thanks
TileDB built with GCS is available on all platforms from conda-forge (https://github.com/conda-forge/tiledb-feedstock/pull/184), as well as linux and macOS wheels installed via pip
from PyPI (Windows should be available in TileDB 2.17). Please let us know if there are any issues.
I'm running on a Mac M2. Is it possible it's something related to tiledb-py for the arm64 architecture not being built with GCS support?
TileDB built with GCS is available on ... macOS wheels installed via
pip
from PyPI. Please let us know if there are any issues.
https://github.com/TileDB-Inc/TileDB-Py/blob/a355893106ac50b4d0d90f492d9bb3a010096919/misc/azure-release.yml#L33-L39
from Feb 2, 2023 https://github.com/TileDB-Inc/TileDB-Py/pull/1614/files#diff-4d387a144a0b721ed0135142faa6865225e425f757cf4d74e3891aae73c8e277R32
@cameronraysmith thanks for the comment. I've opened #1899 to see about (re-)enabling in the wheel builds.
Thank you @ihnorton .