Maksim Kouznetsov
Maksim Kouznetsov
I cannot wait for https://github.com/shirou/gopsutil/pull/1192 (a partial fix for this issue) to be merged. Will it make it into the December release? We are using this package to get the...
I am looking at it now.
So far, I was not able to reproduce this but I will keep trying. Can you check the size of the object on S3 involved in the upload: `ml-checkpoint/51085318-5dd5-45c2-81fd-d3ad495f541c/tensorboard/experiment/729/trial/725/events.out.tfevents.1664468248.exp-729-trial-725-0-729.d7a76451-81d9-49e4-b2b2-61d46293cf29.6.390.0` ?...
2. Yes, I meant upload. I was not able to reproduce your issue with RELEASE.2022-09-25T15-44-53Z of minio. What is your version? On the positive side, assuming that the tfevents size...
Please let us know which version of minio you are using. Until we come up with a more permanent solution, would you mind trying the following workaround: Close the writer...
Did you get a chance to try the workaround?
@sijin-dm , thank you for reporting this. Yes, the partial fix in on master, and that has been released this week (closing the instance of TorchWriter used by our Trial...