GPU utilization bar

[theRealSuperMario]

Sometimes it would be nice to know if the gpu is under-utilized, indicating that the I/O bandwidth limits the GPU.

I am not sure if this is easy to measure, because one might need to figure out if the gpu is idle at some point.

[jhaux]

I agree that this would be super cool to have in the iterator interface, right above the epoch bar for example. But there already is an easy way to do this: Wandb allows you to track system parameters. Sadly it logs all gpus, so you have to take care to only look at the one you are interested in. E.g. if you use CUDA_VISIBLE_DEVICES=4 you need to take a look at the logs under system/gpu.4.<parameter>.