dealbot icon indicating copy to clipboard operation
dealbot copied to clipboard

Support for batch upload of retrieval deals

Open dkkapur opened this issue 4 years ago • 4 comments

The dealbot endpoint supports a list of miners + 1 piece CID to schedule retrievals against.

This works fine in the case of one piece being retrieved from several miners, but in the case that we want to retrieve different pieces (which is usually the case with randomly generated data from the dealbot or for Slingshot data) from different miners at the same time, each retrieval needs to be individually entered in.

Can we instead support a way to upload/list several unique {miner ID, CID} pairs to the endpoint to kick off several retrievals at once?

dkkapur avatar Jul 08 '21 01:07 dkkapur

this is doable. what format do you want to provide the tuples in?

<miner>, <cid> (a CSV), or something else?

willscott avatar Jul 08 '21 15:07 willscott

as discussed today, supporting a JSON POST would work sticking to the format that is currently used to schedule dealbot tasks

dkkapur avatar Jul 09 '21 06:07 dkkapur

P1 scope should just be to have instructions for using JSON POST to support this. P2 scope is to support JSON upload in the UI.

dkkapur avatar Jul 22 '21 23:07 dkkapur

Downgrading this to P3 for now, requires re-prioritization depending dealbot goals in the near term.

dkkapur avatar Nov 23 '21 22:11 dkkapur