issue-tracking icon indicating copy to clipboard operation
issue-tracking copied to clipboard

auto_histogram_gradient_logging on tensorflow datasets

Open kretes opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

Currently (comet 3.23.1) when I specify auto_histogram_gradient_logging=True and/or auto_histogram_gradient_logging when using comet with keras, and using a tf.dataset I see in logs:

COMET WARNING: tensorflow datasets are not currently supported for gradient and activation auto-logging
...
COMET WARNING: auto_histogram_gradient_logging is True, but inputs and targets are not available; unable to log gradients
COMET WARNING: auto_histogram_activation_logging is True, but inputs are not available; unable to log activations

Describe the solution you'd like

  1. This limitation should be clearly stated in the docs https://www.comet.ml/docs/python-sdk/keras/
  2. Some suggested workaround might be published
  3. Is there a chance this is gonna be supported?

Describe alternatives you've considered

I assume I can do a manual sample of the data from the dataset, and pass it to Comet? but if that is so simple - why not implementing that in Comet?

Additional context

kretes avatar Jan 27 '22 12:01 kretes

Hello @kretes. Thank you for catching this. Would you be able to share a code snippet for how you're logging your gradients and activations with tf.dataset?

DN6 avatar Feb 03 '22 21:02 DN6

I'm not doing it at the moment. I just assumed that one can do sth like (psuedocode) inputs, targets = zip(*ds.take(10)) and pass this to Experiment, but I don't see any parameters like this in https://www.comet.ml/docs/python-sdk/Experiment/#experiment So this more a question of how should this be done ;]

kretes avatar Feb 25 '22 10:02 kretes

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Oct 19 '23 21:10 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Oct 24 '23 21:10 github-actions[bot]