Chen Qian

@andreped Thanks for reporting the issue! Currently users would need to write their own custom training loop to handle the gradient accumulation, which is not too hard, so we have...

Thanks all for the great discussion! @andreped Thanks for raising the BN issue, yes, it's something we should support. Actually I am curious about the performanceloss if we don't handle...

@jouvetg Thanks for reporting the issue! We will add this optimizer to our monitoring list, once it gets a decent number of interest and usage, we will offer it in...

Sorry for missing this issue (somehow it slipped away). I like the idea of flexible dtype, actually in the [experimental Keras optimizer](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/experimental/Optimizer#add_variable), there is a method `add_variable` that suits your...

@adriangb Thanks for reporting the issue! There has not been any change on `get_weights()` for months. For loading optimizer weights, please make sure you call `load_weights()` if you want to...

`get_weights` method is removed in the new optimizer, if you need to access the weights, please call `variables()` method. But I don't think the issue is caused by this deprecation,...

There is not `set_weights` method. The current workaround is to loop over the variable list and set each variable individually. Some more context - we no longer keep the `set_weights`...

You need to call `load_weights()` to restore the weights, otherwise it will be lazily loaded to my knowledge. For your code snippet, you want to do `new.load_weights("model")` after the `load_model()`...

``` import tensorflow as tf model = tf.keras.Sequential( [ tf.keras.Input(shape=(1,)), tf.keras.layers.Dense(1, activation="softmax"), ] ) model.compile(optimizer="adam", loss="categorical_crossentropy") model.fit([[1]], [0]) model.save("model") new = tf.keras.models.load_model("model") new.load_weights("model") ``` It loads the optimizer state. You...

@adriangb What's your TF and Keras version? The snippet works as expected on my testing.