Tensorboard callback is blocking process

[rivershah]

I am unable to find the transferred issue: https://github.com/keras-team/tf-keras/issues/496 This issue is still occuring and creates performance bottleneck when writing to cloud storage

[mattdangerw]

The only thing I could see that we might be able to control from the Keras side is, max_queue in create_file_writer. We could try setting that to a larger value here https://github.com/keras-team/keras/blob/19ba3d1dd6079c4b72cc68b8ae969f5da3e09ea5/keras/src/callbacks/tensorboard.py#L229-L243 (are you able to try that out?).

The comment here makes it seem like this might be a issue with the tensorflow summary ops. If that's the case, we probably need to open up an issue with tensorflow to get this resolved. Or see if one already exists.

[mattdangerw]

@grasskin cc'ing you here as well in case you have more context.