exclude_from_weight_decay in RectifiedAdam()

[markub3327]

Describe the feature and the current behavior/state.

Can RectifiedAdam have same parameter exclude_from_weight_decay as AdamW? We must exclude some parameters of the model from weight decay.

Relevant information

• Are you willing to contribute it (yes/no): yes If you wish to contribute, then read the requirements for new contributions in CONTRIBUTING.md
• Are you willing to maintain it going forward? (yes/no): yes
• Is there a relevant academic paper? (if so, where): no
• Does the relavent academic paper exceed 50 citations? (yes/no): no
• Is there already an implementation in another framework? (if so, where): yes
• Was it part of tf.contrib? (if so, where): No

Which API type would this fall under (layer, metric, optimizer, etc.)

optimizer

Who will benefit with this feature?

Everyone

Any other info.

Thanks a lot.

[bhack]

Probably this component is orphan. Gently ping @szutenberg

@markub3327 Can you check with a ticket on https://github.com/keras-team/keras if they are interested if you port this in Keras with your exclude_from_weight_decay extension?

[markub3327]

@bhack The ticket is done.

[bhack]

Thank you as AdamW was just duplicated there 27 days ago probably you could add a comment there also for exclude_from_weight_decay for AdamW

[markub3327]

@bhack I have done it.

Please look here.

[seanpmorgan]

TensorFlow Addons is transitioning to a minimal maintenance and release mode. New features will not be added to this repository. For more information, please see our public messaging on this decision: TensorFlow Addons Wind Down

Please consider sending feature requests / contributions to other repositories in the TF community with a similar charters to TFA: Keras Keras-CV Keras-NLP