[fix] Add first draft of the PR for issue#349

[nabenabe0928]

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)

Note that a Pull Request should only contain one of refactoring, new features or documentation changes. Please separate these changes and send us individual PRs for each. For more information on how to create a good pull request, please refer to The anatomy of a perfect pull request.

Checklist:

• [x] My code follows the code style of this project.
• [x] My change requires a change to the documentation.
• [x] I have updated the documentation accordingly.

• [x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• [x] Have you added an explanation of what your changes do and why you'd like us to include them?
• [ ] Have you written new tests for your core changes, as applicable?
• [x] Have you successfully ran tests with your changes locally?

Description

See issue#349. Note that although this PR has so many changes, most changes are gathered in three files:

1. tae.py
2. evaluator.py
3. abstract_evaluator.py

The changes in other files are mostly for tests or deletions because I integrated those features into the aforementioned three files.

[codecov[bot]]

Codecov Report

Merging #365 (180ff33) into development (a679b09) will increase coverage by 1.85%. The diff coverage is 99.33%.

@@               Coverage Diff               @@
##           development     #365      +/-   ##
===============================================
+ Coverage        83.44%   85.30%   +1.85%     
===============================================
  Files              163      163              
  Lines             9634     9567      -67     
  Branches          1689     1665      -24     
===============================================
+ Hits              8039     8161     +122     
+ Misses            1114      953     -161     
+ Partials           481      453      -28     

Impacted Files	Coverage Δ
autoPyTorch/api/tabular_classification.py	90.90% <ø> (ø)
autoPyTorch/api/tabular_regression.py	100.00% <ø> (ø)
autoPyTorch/pipeline/base_pipeline.py	71.13% <ø> (-0.95%)	:arrow_down:
autoPyTorch/datasets/resampling_strategy.py	91.48% <80.00%> (-0.65%)	:arrow_down:
autoPyTorch/api/base_task.py	84.06% <90.90%> (-0.34%)	:arrow_down:
autoPyTorch/evaluation/evaluator.py	99.01% <99.01%> (ø)
autoPyTorch/evaluation/abstract_evaluator.py	99.14% <99.49%> (+24.49%)	:arrow_up:
...utoPyTorch/evaluation/pipeline_class_collection.py	100.00% <100.00%> (ø)
autoPyTorch/evaluation/tae.py	95.69% <100.00%> (+25.05%)	:arrow_up:
autoPyTorch/evaluation/utils.py	85.55% <100.00%> (+11.94%)	:arrow_up:
... and 14 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a679b09...180ff33. Read the comment docs.