Training code can't train adf network

Hi, I found a problem while training adf network. If the initial learning rate lr in the code is used is 0.1 when the training loss is nan, the adf network can't be trained. I would like to ask what to do about this problem to get the adf network trained?

Hi, I found a problem while training adf network. If the initial learning rate lr in the code is used is 0.1 when the training loss is nan, the adf network can't be trained. I would like to ask what to do about this problem to get the adf network trained?

I'm sorry to interrupt, I just want to know how the ADF network was trained to get it, because I found that I applied the adf to other networks performance is very poor, worse than the baseline