FEDOT icon indicating copy to clipboard operation
FEDOT copied to clipboard

Refactor metric evaluation for tuner

Open YamLyubov opened this issue 2 years ago • 1 comments

Tuner now uses PipelineObjectiveEvaluate to evaluate metric while hyperparameters optimization.

  • Tuner uses the same metric function as composer
  • Pipeline.fine_tune_all_nodes is removed. To tune pipeline one can use PipelineTuner.tune. Initialized once, tuner can be used for tuning different pipelines, not only one.
  • Functions calculate_loss_function, cv_tabular_predictions, cv_time_series_predictions are replaced by PipelineObjectiveEvaluate and removed.
  • HyperoptTuner does not use information about whether metric is minimized or maximized. It always minimizes the metric.

#788

YamLyubov avatar Aug 12 '22 09:08 YamLyubov

Hello @YamLyubov! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2022-09-08 13:33:37 UTC

pep8speaks avatar Aug 12 '22 09:08 pep8speaks

Codecov Report

Merging #818 (6bcaff5) into master (c8eed58) will decrease coverage by 0.26%. The diff coverage is 98.40%.

@@            Coverage Diff             @@
##           master     aimclub/FEDOT#818      +/-   ##
==========================================
- Coverage   87.87%   87.60%   -0.27%     
==========================================
  Files         196      196              
  Lines       13405    13332      -73     
==========================================
- Hits        11780    11680     -100     
- Misses       1625     1652      +27     
Impacted Files Coverage Δ
fedot/core/pipelines/tuning/sequential.py 95.23% <96.42%> (-0.42%) :arrow_down:
fedot/core/pipelines/tuning/tuner_builder.py 97.01% <97.01%> (ø)
fedot/api/api_utils/api_composer.py 98.70% <100.00%> (+3.55%) :arrow_up:
fedot/api/time.py 96.77% <100.00%> (+0.16%) :arrow_up:
fedot/core/composer/composer_builder.py 90.67% <100.00%> (-0.53%) :arrow_down:
fedot/core/composer/metrics.py 97.25% <100.00%> (-0.02%) :arrow_down:
fedot/core/operations/atomized_model.py 84.37% <100.00%> (+1.04%) :arrow_up:
...t/core/optimisers/objective/data_objective_eval.py 92.40% <100.00%> (+0.19%) :arrow_up:
fedot/core/pipelines/pipeline.py 96.87% <100.00%> (+0.40%) :arrow_up:
fedot/core/pipelines/tuning/tuner_interface.py 98.21% <100.00%> (+22.77%) :arrow_up:
... and 14 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Aug 19 '22 10:08 codecov[bot]

Еще пофикси pep8 замечания выше, их много, тк сделали правила более строгими

gkirgizov avatar Sep 06 '22 10:09 gkirgizov