Dhruv Nair
Dhruv Nair
@mrbraden56 Would you be able to share the script you're using to actually start the training run? Also would it be possible to run the script in debug mode. You...
The file should be created in the current working directory and will be called `comet.log`
Hi @george-gca. I was able to reproduce this. I've raised the issue with our engineering team and it has been prioritized.
Thanks for reporting @mova. Would you be able to link me to the experiment with the issue? I can't seem to find it in your project page.
Thank you @mova! I was able to reproduce the issue. Let me check with our team to see what might be happening. I will update you here when we have...
@mova Would it be possible to increase the character length of your hash strings so that they do not look like scientific notation? This could be a quick way to...
Hello @kretes. Thank you for catching this. Would you be able to share a code snippet for how you're logging your gradients and activations with `tf.dataset`?
Hi @florianblume. You should be able to delete logged assets (images, checkpoints, etc) via the [Comet API Object.](https://www.comet.ml/docs/python-sdk/APIExperiment/#apiexperimentdelete_asset). Here is a snippet to delete an asset from an experiment based...
Hello @nsriniva03. Looking into this. Just to confirm, are you running distributed training with Comet? Would it be possible to share the code used to run these experiments? It would...
Thank you! That would be very helpful. Additionally, are you using data parallel training? If so, is your data being distributed the same way across your training nodes? Or is...