lmeyerov

Results 264 comments of lmeyerov

@hayesgb Digging a bit more, switching to `asynchronous=True ... await fs._isfile(existing_file_path)` does not work around the issue: the warning still triggers and the wrong result still gets returned

@hayesgb (Continuing from https://github.com/dask/adlfs/issues/261) Just tried from head: - [ ] existing files `isfile()` is quickly & **incorrectly** returning `False`; no async warning anymore - [ ] existing dirs `isdir()`...

Also if it helps, my paths look like: `abfs://somecontainer/mydata/mydata2/myfile`

`AttributeError: 'AzureBlobFileSystem' object has no attribute 'details'`

FYI, having more luck with variants of: ```python async def aexists_dir(path): blob_service_client = BlobServiceClient.from_connection_string(conn_str) async with blob_service_client: container_client = blob_service_client.get_container_client(az_storage_container_name) async for myblob in container_client.list_blobs(name_starts_with=path): return myblob['name'] != path return...

```python { "metadata": None, "creation_time": datetime.datetime(2020, 9, 29, 0, 16, 6, tzinfo=datetime.timezone.utc), "deleted": None, "deleted_time": None, "last_modified": datetime.datetime(2021, 8, 13, 15, 35, 35, tzinfo=datetime.timezone.utc), "content_settings": { "content_type": "application/x-gzip", "content_encoding": None,...

Yes - it's a potentially big folder (named parquet dumps), in this case I wouldn't be surprised if 1K-10K files. I think async list_files paginates, though I'm unsure of how...

Sure -- will check on Th/F (am traveling) At the same time, if anything around async multi-connection downloads of indiv + folder blobs, happy to check there. Currently investigating how...

1. Currently single-node / multicore . Our Azure GPU VMs have something like 2-8 NICs with 8-32 Gbps, and I think AWS/GCP end up similar, so focusing on saturating abfs...

Hmm, this makes me unsure about safety in a concurrent environment. Both in general, and especially for async=True, how should we think about it? * Any file system / fsspec...