Entrypoint support for abfss in adlfs prevent this protocol to be overwritten by another package

[pierre-godard]

Hi!

Sorry if this issue is misplaced and should go to the adlfs package instead. I already created an issue there, but after thinking of it, maybe it is more relevant to post it here.

You'll find all the details in this issue: fsspec/adlfs#302

I'm not sure if I should copy/paste the whole issue here or if the link is enough, let me know!

[martindurant]

(the link is fine)

It wasn't really anticipated that there would be multiple implementations of a given protocol, although "ssh" has, and although Intake deals with a similar plugin problem. I think that in your case, you might consider including a call to fsspec.registry.register_implementation in your code and require users to import your package if they want the non-default backend.

For the more general issue:

• I don't know how entrypoints decides on the order of hits, but suspect it might just be the native filesystem order of directories. That means that the specific installation steps would matter, and no general rule like "first" or "last" will suffice.
• really, this sounds like a configuration thing. I think it would make sense to optionally add to known_implementations via the config, and that these take priority (and calls to register_implementation are highest priority).

[KeerthiYandaOS]

@martindurant Looks like the implementation (register_implementation) invoked "last" is taking precedence over the previous one. As the entry point list is in below order:

[EntryPoint(name='abfs', value='mypackage.AzureBlobFileSystem', group='fsspec.specs'),
 EntryPoint(name='abfss', value='mypackage.AzureBlobFileSystem', group='fsspec.specs'),
 EntryPoint(name='abfss', value='adlfs.AzureBlobFileSystem', group='fsspec.specs')]

As the adlfs.AzureBlobFileSystem is getting registered at the end, the request for abfss protocol is reaching the adlfs.AzureBlobFileSystem implementation instead of the mypackage.AzureBlobFileSystem

@pierre-godard were you able to find any alternate solution? it will be helpful if you can share your findings.

Unfortunately, I can't use the register_implementation as it will be lost once the package is re-installed in my scenario.

[KeerthiYandaOS]

Created a pull request to avoid the entry-point overriding: https://github.com/fsspec/filesystem_spec/pull/1389