regularization amplitude tuning runs very slowly

[megbedell]

It seems like there's some sort of memory leak (TensorFlow not fully clearing the graph?) that means every subsequent order runs slower than the previous one.

Intermediate recommended fix is to run orders in small batches.

[NairdaK]

Hi, for me the issue in this contextit is actually not the speed that suffers, but the memory that is used up. Order after order memory is accumulated (and not released), so that for typical data sets with around 100 epochs my machine is using up all the memory, and the process is freezed.

Edit: Actually, the same happens in the RV computation, but is a more severe problem with the reg. parameters optimization.