pyod icon indicating copy to clipboard operation
pyod copied to clipboard

Refactored PyTorch Autonencoder built up (Proposal to solve #434)

Open Lucew opened this issue 2 years ago • 3 comments

This Pullrequest is a proposal to fix #434.

I refactored the setup of the Autoencoder (PyTorch Version) in order to address the points of #434. The order of layers is now consistent with the Tensorflow implementation.

The order of Neurons->BatchNorm->Activation follows the Standard-Implementation of ResNet

We adopt batch normalization (BN) [16] right after each convolution and before activation, following [16].

The printed Network now looks like:

AutoEncoder(batch_norm=True, batch_size=32, contamination=0.1,
      device=device(type='cpu'), dropout_rate=0.2, epochs=10,
      hidden_activation='relu', hidden_neurons=[64, 32],
      learning_rate=0.001, loss_fn=MSELoss(), preprocessing=True,
      weight_decay=1e-05)
InnerAutoencoder(
  (activation): ReLU()
  (encoder): Sequential(
    (linear0): Linear(in_features=300, out_features=64, bias=True)
    (batch_norm0): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu0): ReLU()
    (dropout0): Dropout(p=0.2, inplace=False)
    (linear1): Linear(in_features=64, out_features=32, bias=True)
    (batch_norm1): BatchNorm1d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu1): ReLU()
    (dropout1): Dropout(p=0.2, inplace=False)
  )
  (decoder): Sequential(
    (linear0): Linear(in_features=32, out_features=64, bias=True)
    (batch_norm0): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu0): ReLU()
    (dropout0): Dropout(p=0.2, inplace=False)
    (linear1): Linear(in_features=64, out_features=300, bias=True)
    (relu1): ReLU()
  )
)

All Submissions Basics:

  • [x] Have you followed the guidelines in our Contributing document?
  • [x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • [x] Have you checked all Issues to tie the PR to a specific one?

All Submissions Cores:

  • [x] Have you added an explanation of what your changes do and why you'd like us to include them?
  • [x] Have you written new tests for your core changes, as applicable?
  • [x] Have you successfully ran tests with your changes locally?
  • [x] Does your submission pass tests, including CircleCI, Travis CI, and AppVeyor?
  • [x] Does your submission have appropriate code coverage? The cutoff threshold is 95% by Coversall.

Lucew avatar Sep 03 '22 11:09 Lucew

Pull Request Test Coverage Report for Build 2984228284

  • 19 of 19 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.0007%) to 93.797%

Totals Coverage Status
Change from base Build 2980030757: 0.0007%
Covered Lines: 7848
Relevant Lines: 8367

💛 - Coveralls

coveralls avatar Sep 03 '22 11:09 coveralls

sorry there will be some delay on my end to multiple ddl...will review very soon!! thanks

yzhao062 avatar Sep 14 '22 13:09 yzhao062

Yeah, no worries. I know that life happens sometimes. Just let me know, once I can do something to bring this further!

Lucew avatar Sep 14 '22 20:09 Lucew

perfect it is in! I will make some local minor updates as needed :) thanks!!

yzhao062 avatar Sep 23 '22 13:09 yzhao062