AxFoundation
S3 protocol
What is the problem / what does the code in this PR do
Adds S3 functionality to strax
Can you briefly describe how it works?
We can now use an S3 storage system to store data
Can you give a minimal working example (or illustrate with a figure)?
import strax
st = strax.Context()
s3_storage = strax.S3Frontent()
st.storage = [s3_storage]
Please include the following if applicable:
- Update the docstring(s)
- Update the documentation
- Tests to check the (new) code is working as desired.
- Does it solve one of the open issues on github?
Please make sure that all automated tests have passed before asking for a review (you can save the PR as a draft otherwise).
Thanks, @LuisSanchez25. Would it be more appropriate to put these new StorageFrontend and StorageBackend to straxen like RucioRemoteFrontend https://github.com/XENONnT/straxen/blob/a412b4382eed3277fd599c105d69855df38bc7f8/straxen/storage/rucio_remote.py#L29?
And I generally think this is a super good idea! We should benefit from the market resources!
Hey @dachengx I am a bit unfamiliar with why the decision was made to put the Rucio frontend in straxen rather than in strax, that feels like a tool that would be beneficial for others outside of XENON right? So to me it might make more sense to move the Rucio storage to straxen but maybe I am just missing something.
Thanks! I am still in the process of fully testing this, I think it still needs some tweaks but I should have a working prototype soon!
@LuisSanchez25 I think by design, strax is for the prototypes of all classes, like plugins and storage. straxen will inherit the classes and make the functions more specific. I think this is why they put the RucioRemoteFrontend to straxen.
coverage: 87.217% (-1.7%) from 88.958% when pulling 4bb5ddbe8687fc33b794e155bf17e6ca323b30e7 on s3_protocol into cd79adc2284715dfd44bca15067829512f16b03f on master.
