matorage
matorage copied to clipboard
When there are many cores during multi-processing, the metadata and the actually uploaded file are not synced.
I will write test code related to this and conduct experiments.
To implement it correctly, you should check https://github.com/graykode/matorage/blob/master/matorage/uploader.py#L41 to see if the upload is a success(200) code and put it in metadata.json only when it succeeds.