Scalars and plots are not uploaded when 'Git diff to large to store'

[gorogm]

I have a quite simple setup, running a training from a jupyter notebook. I'm experiencing with clearml 1.4 and 1.6 as well, that whenever git diff is too large: ======> WARNING! Git diff to large to store (824Kb), skipping uncommitted changes <====== then neither plots nor scalars generated during training are appearing at clear.ml website, which means the results are lost...

A 800Kb git diff can very easily happen. How to overcome this issue? Thanks.

[jkhenning]

Hi @gorogm ,

When running a local server, I think this limitation can be updated (but due to a limitation of MongoDB, the diff size cannot be too large anyway). In app.clear.ml this limitation is also enforced to protect scale and to allow good quality of service. Can I ask why do you have such large diffs?