issue-tracking
issue-tracking copied to clipboard
auto_histogram_gradient_logging on tensorflow datasets
Is your feature request related to a problem? Please describe.
Currently (comet 3.23.1) when I specify auto_histogram_gradient_logging=True
and/or auto_histogram_gradient_logging
when using comet with keras, and using a tf.dataset I see in logs:
COMET WARNING: tensorflow datasets are not currently supported for gradient and activation auto-logging
...
COMET WARNING: auto_histogram_gradient_logging is True, but inputs and targets are not available; unable to log gradients
COMET WARNING: auto_histogram_activation_logging is True, but inputs are not available; unable to log activations
Describe the solution you'd like
- This limitation should be clearly stated in the docs https://www.comet.ml/docs/python-sdk/keras/
- Some suggested workaround might be published
- Is there a chance this is gonna be supported?
Describe alternatives you've considered
I assume I can do a manual sample of the data from the dataset, and pass it to Comet? but if that is so simple - why not implementing that in Comet?
Additional context
Hello @kretes. Thank you for catching this. Would you be able to share a code snippet for how you're logging your gradients and activations with tf.dataset
?
I'm not doing it at the moment. I just assumed that one can do sth like (psuedocode) inputs, targets = zip(*ds.take(10))
and pass this to Experiment
, but I don't see any parameters like this in https://www.comet.ml/docs/python-sdk/Experiment/#experiment
So this more a question of how should this be done ;]
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
This issue was closed because it has been stalled for 5 days with no activity.