Performance issue about in TF2/utils.py

[DLPerf]

Hello! Our static bug checker has found a performance issue in TF2/utils.py: trainer is repeatedly called in a for loop, but there is a tf.function decorated function optimize_one_step defined and called in trainer.

In that case, when is called in a loop, the function will create a new graph every time, and that can trigger tf.function retracing warning.

Here is the tensorflow document to support it.

Briefly, for better efficiency, it's better to use:

@tf.function
def inner():
    pass

def outer():
    inner()  

than:

def outer():
    @tf.function
    def inner():
        pass
    inner()

Looking forward to your reply.

[DLPerf]

Just taking the inner function outside would help! Btw, I'm glad to make a pr if you are too busy. @PwnerHarry