boinc icon indicating copy to clipboard operation
boinc copied to clipboard

Support for S3/Ceph direct uploads.

Open lfield opened this issue 6 years ago • 3 comments

For some projects data management could be improved by uploading files directly into an S3/Ceph storage system close to the volunteer. This can be done with a presigned URL after authentication. The data server protocol could be updated to send back to the client a direct URL where to put the file. More details can be found in the following paper.

Contact point: @AenBleidd

lfield avatar Oct 15 '18 12:10 lfield

The paper's link is not correct.

adamradocz avatar Oct 15 '18 13:10 adamradocz

Should work now

lfield avatar Oct 15 '18 13:10 lfield

Oh - I like this presigned URL mechanism - I like this idea since I think that shifting to using object storage will be an important thing in the future. In fact it would be great if everything from create_work, downloads, uploads, validators and assimilators could use the object storage. This would greatly simplify project storage.

TheAspens avatar Nov 01 '18 18:11 TheAspens