keras-tqdm
keras-tqdm copied to clipboard
on_batch_end incredibly slow
When I run an epoch with about 2 million samples and a batch size of 1, Keras/Tensorflow finishes in about 45 minutes.
When I run that same epoch, with TQDMCallback, it takes over 13 hours.
It seems like there's logging going on during batches by default, which shouldn't be default behaviour. Not sure if that entirely explains it, though.
Same issue here. The problem seems to be the call to np.mean(logs[metric], axis=None)
inside the format_metrics
function. When you're training 100 batches per second that log gets big fast.