File client for reading from S3 Object Storage

[levan92]

Hi! I am directed here from mmdetection. It seems like petrel_client is something internal to your lab. Are there plans to support a file client for S3 Object Storage?

[zhouzaida]

Hi @levan92 , yet, the petrel_client is a backend for internal use. In fact, we have a plan to support a file client for S3 object storage next month. Of course, you can also refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/fileio/file_client.py#L85 to implement it if you are interested.

[levan92]

Sure, will an implementation that utilises boto3 work?

[zhouzaida]

Sure, will an implementation that utilises boto3 work?

sure

[zhouzaida]

The issue will be resolved by #1545

[jckhang]

Is there any plan on adding this feature soon?

[mmeendez8]

Can we get any info about this? Is there any problem with current work? Willing to help, this would be an awesome feature!

[sandeepnmenon]

+1 When I provide s3 path I get the error

  File "/home/s.menon//bazel-bin/source/offboard/ml/third_party/bevfusion/train.runfiles/offboard_python_analysis_mmcv_full/site-packages/mmcv/fileio/file_client.py", line 116, in __init__
    from petrel_client import client
ModuleNotFoundError: No module named 'petrel_client'

Is this the package? https://pypi.org/project/petrel/ Or is it something internal

[matthost]

Is this still a problem in mmdet 3.x?