André Pedersen

> @andreped Nothing to share yet. In the meantime, are you aware of [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners)? That could be a decent stopgap if you need to test against Windows ASAP....

> @andreped When you say a "standalone application", are you talking about a [standalone application built via MATLAB Compiler](https://www.mathworks.com/help/compiler/standalone-applications.html?s_tid=CRUX_lftnav) (i.e. an application that does not require the target machine to...

@mcafaro I see... I was hoping that it was possible, or at least in the near future, to be able to use our own MATLAB license when doing CI with...

> Could you please specify the use cases for this feature. What do you mean by "use cases"? Do you mean scenarios on which having a simple way to perform...

> Currently users would need to write their own custom training loop to handle the gradient accumulation Actually, you don't even need to write your own custom training loops anymore...

Also note that if you introduce gradient accumulation naively, like I did above, then some layers will not be directly compatible. You will haven suboptimal behaviour on **BatchNormalization**, for instance,...

> Actually I am curious about the performanceloss if we don't handle the accumulation for BN layer @chenmoneygithub I have not performed a rigorous test to benchmark w/wo BN with...

Just mentioning that I have a stable implementation for gradient accumulation now, as a temporary solution until Keras adds a proper method for it: https://github.com/andreped/GradientAccumulator - Simply wrap the model...

> I came across here looking for gradient accumulation where I will train using: > > 1- Multiple GPUs. > > 2- FP16 > > 3- Functional API. > >...

@meliksahturker If you follow the commit history of the tool you will see that I used the code you mentioned as baseline. However, I did not reach the same results...