mlem
mlem copied to clipboard
Issue listing MLEM objects on S3
Using the latest release 0.2.7
, I have initialized and imported a model and as a target, used a S3 bucket (mlem init s3://bucket/awesome_project
followed by mlem import ckpt/best.pkl s3://bucket/awesome_project/ckpt/best
).
Then, when I try to list the models in that S3 buckets, I get the following fatal error (mlem ls s3://bucket/awesome_project
):
❌ Unexpected error: _find() got an unexpected keyword argument 'recursive'
Please report it here: <https://github.com/iterative/mlem/issues>
From my understanding and digging inside mlem's code, it relies on fsspec
's glob method. However, when creating a S3 location, its fs.glob
method doesn't support the recursive
kwargs
(https://github.com/iterative/mlem/blob/HEAD/mlem/core/index.py#L86-L88), which is relayed to the fsspec
's ls
method (https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.glob).
Not sure why the recursive
is needed when the pattern has that **
.
PS. having the latest fsspec
: 2022.7.1
and s3fs
: 2022.7.1
Happy to help if I wasn't clear.