adlfs

adlfs copied to clipboard

3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05) \n[GCC 9.3.0] Name: fsspec Version: 2021.6.1 The `rm` function will accept a list of file paths, therefore it should...

MrNickArcher

3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05) \n[GCC 9.3.0] Name: fsspec Version: 2021.6.1 1. Open Azure Synapse, 2. Create python notebook 3. Execute the following code...

MrNickArcher

first run of Parquet schema read much slower than BlobClient?

9

First run this took 40 seconds + ```{python} #slow with abfs.open('example-container/test.parquet') as f: schema = pq.read_schema(f) ``` This which I'm pretty sure downloads the entire blob takes under half a...

astrowonk

When running `pytest` with `azure-storage-blob==12.14.0`, we see the following error: ``` self = async def _initial_request(self): range_header, range_validation = validate_and_format_range_headers( self._initial_range[0], self._initial_range[1], start_range_required=False, end_range_required=False, check_content_md5=self._validate_content) try: location_mode, response = await...

hayesgb

Revert "Support account_host and check blob size attribute exists"

1

Reverts fsspec/adlfs#341

hayesgb

Make DefaultAzureCredential as default credential method instead of anonymous access?

5

Hi there what do you think of making DefaultAzureCredential as the default credential method instead of anonymouns access because I think that is the most common use case.

QianqianNie

Make AzureBlobFileSystem anon behaviour configurable via env var.

5

Making the `anon` default value configurable from an env var makes integration easier with helm charts that provide features like RBAC in Azure accounts. In a multi-cloud application I am...

microft

Support DataLakeServiceClient for Azure Gen 2 Storage

5

Currently, adlfs is using BlobServiceClient for both ADLS Gen 2 and Blob storage. Blob client is using blob apis underneath but if we want to use data lake apis currently...

KeerthiYandaOS

document that `credential` needs to be from azure.identity.aio

1

if it's a sync credential, then the weakref.finalize callback will cause an error, `close() -> None` cannot be awaited.

temporaer

Issue with parallel uploads to the same blob

1

There seems to be an issue when 2 instances of this file system write to the same blob from 2 different processes in parallel, where one of the uploads fails...

M0dEx