Make DefaultAzureCredential as default credential method instead of anonymous access?

[QianqianNie]

Hi there

what do you think of making DefaultAzureCredential as the default credential method instead of anonymouns access because I think that is the most common use case.

[agrinh]

+1, that would be great.

[TomAugspurger]

xref https://github.com/fsspec/adlfs/pull/250, where this was implemented but then reverted. I think we can do this, but we'll need a deprecation cycle.

I was somewhat worried that this would be too noisy of a deprecation, but if we think that the majority of users are specifying credential anyway (which I think they are) then the only people affected are those relying on credential="anon".

So +1 to deprecating the current default with a FutureWarning and then moving to DefaultAzureCredential in a later future.

[edit]: One slight modification: this would need to interact carefully with a SAS token in the URL. From the azure docs:

The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential

• except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError. If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key" should be the storage account key.

[JoostvDoorn]

Is there any way to move this forward? There are some libraries that do not expose fsspec storage options because they assume the anon=False logic like in s3, and I suppose azure is not their main target audience. As a workaround would it be possible to add an env option to disable anon access before we transition to anon=False by default?

[agrinh]

I'm with @JoostvDoorn, this is a bit of a headache. There needs to be some mechanism to override this without explicit access to the storage options until the default can be changed. Either an env variable or some override in the URL itself.

[JoostvDoorn]

For those interested in this https://github.com/fsspec/filesystem_spec/pull/1194 provides a generic alternative solution (make sure to update fsspec as it's pretty recent addition). You can pass a default argument to FSSPEC for specific implementations through an env variable as follows:

os.environ["FSSPEC_ABFS"] = """
{
  "anon": false
}
"""

I'll be closing my PR (https://github.com/fsspec/adlfs/pull/393) as it's not getting any traction.