Autoencoder learning rate

[msho-nb]

For the autoencoder in pyod, how do I adjust the learning rate?

[yzhao062]

you make a legit point. I just realize we simply use the default optimizer's learning rate. will relax that...

[msho-nb]

Have this been resolved yet ? :)

[amsehili]

@msho-nb how about passing an optimizer instance instead of a string?

from tensorflow.keras.optimizers import Adam                                                                                   
optimizer = Adam(learning_rate=0.01)
ae = AutoEncoder(..., optimizer=optimizer)

[Lucew]

General Opinion

Just looking at the implementation for Autoencoders (Variational-Autoencoder, Autoencoder, AutoencoderTorch):

Learning rate adaptation should be possible for all of those. How one would do that differs between the PyTorch and Tensorflow implementation.

For Tensorflow: Instead of passing a string element, one would pass an Instance of https://keras.io/api/optimizers/, where you could define your learning rate while instantiating the optimizer like: e.g. modified from examples/auto_encoder_example.py Line 38 clf = AutoEncoder(epochs=30, contamination=contamination, optimizer=keras.optimizers.Adam(learning_rate=0.01))

For PyTorch: Here you can directly specify the learning rate in the creation of the Autoencoder (from pyod.models.auto_encoder_torch import AutoEncoder) e.g. modified from examples/auto_encoder_torch_example.py Line 40 clf = AutoEncoder(epochs=10, learning_rate=0.01) Unfortunately, this feature is not yet documented in the Docstring of the AutoEncoder class.

Comment

Maybe I missed the update in the Git history and this Issue was already fixed. I'm just looking for things to contribute and recently found this issue.

Proof:

Variational AutoEncoder:

https://github.com/yzhao062/pyod/blob/002722132b0aec7519cf8f9765754c5d12e20050/pyod/models/vae.py#L100-L102 https://github.com/yzhao062/pyod/blob/002722132b0aec7519cf8f9765754c5d12e20050/pyod/models/vae.py#L300 https://keras.io/api/optimizers/

AutoEncoder: https://github.com/yzhao062/pyod/blob/002722132b0aec7519cf8f9765754c5d12e20050/pyod/models/auto_encoder.py#L59-L61 https://github.com/yzhao062/pyod/blob/002722132b0aec7519cf8f9765754c5d12e20050/pyod/models/auto_encoder.py#L194

AutoEncoder PyTorch: https://github.com/yzhao062/pyod/blob/002722132b0aec7519cf8f9765754c5d12e20050/pyod/models/auto_encoder_torch.py#L211-L217

[yzhao062]

Hi Lucew,

Thanks for the thought on this! Happy to take in a PR to improve the current AE/VAE.