xloya

Tesorflow and Pytorch Lighting natively support fsspec-compatible file systems: 1.https://github.com/tensorflow/tensorboard/pull/5248 2.https://lightning.ai/docs/pytorch/stable/common/remote_fs.html

> @xloya do you have some documents about fsspec, maybe we should discuss more about the pros and cons of fsspec, pyarrow filesystem and others. Yeah, I will post a...

@jerryshao @noidname01 @coolderli Hi, I have open a document for implementation selections. Please take a look and comment if you have any questions. https://docs.google.com/document/d/1y1GX7HWha1DH6XFU7VC_0NzmOoTA9j3Ppw1yEzjPpf8/edit?usp=sharing

As far as I know, JuiceFS community version provides Hadoop SDK, please refer to: https://juicefs.com/docs/zh/community/hadoop_java_sdk. So I think JuiceFS can be supported on Fileset using Hadoop SDK like S3.

@jerryshao @yuqi1129 @mchades Could you help me review this PR when you have time? Thanks a lot.

> @xloya is it ready for review? Yeah, the code have been changed according to the discussion.

> Hi @xloya This PR is a little big. I would suggest to split into several small PRs. For this PR, I think we can focus on the definition and...

@jerryshao I have split the code, this PR only contains some code skeleton in the Server / Java Client, and some base classes. I will add the interface implementations, UTs...

> Overall LGTM, can you please do as I mentioned? Sure, I will finish it today.

@jerryshao Hi, since some classes in `Core` / `Catalog` modules involved in this PR currently implement the `FilesetCatalog` interface, I can only use `throw new UnsupportedOperationException("Not implemented")` to handle method...