FEDOT icon indicating copy to clipboard operation
FEDOT copied to clipboard

Published 20 hours ago • verified publisher icon aimclub

Operation repo refactor

Open kasyanovse opened this issue 1 year ago • 4 comments

This is a 🔨 code refactoring.

Summary

Рефактор тегов и пресетов:

  1. Разделение. Теперь пресеты никак не пересекаются с тегами и служат лишь как апи.
  2. Использование Enum везде, где только можно.
  3. Контроль совпадения тегов в репозитории и в коде.
  4. Упрощение кода, в связи с чем стало проще добавлять новые виды моделей.

Главная задача - утвердить или отклонить предлагаемые изменения. В связи с их большим количеством, предлагаю сначала сделать ревью этого PR, потом доработать в других PR, но в мастер влить одновременно все доработки.

В отдельных PR будет доработано:

  1. Пресет GPU. Сейчас не работает так как надо.
  2. Удален пресет TREE. Сейчас он в коде есть, но по факту его нет. Нужно вычищать код и тесты от него.
  3. TODO.
  4. Будут добавлены новые пресеты, которые будут выбираться автоматически при использовании PresetsEnum.AUTO (выбор моделей в зависимости от времени, выделенного на композицию, и размера данных так, чтобы в композицию не попали слишком требовательные по времени модели).
  5. Фикс тегов atomized_model. Сейчас это просто костыль.

После завершения этого цикла PR добавлять новые типы моделей и теги станет гораздо проще. До этого добавление неминуемо вело к усложнению и еще большему запутыванию системы пресетов/тегов.

kasyanovse avatar Jan 19 '24 15:01 kasyanovse

Hello @kasyanovse! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 43:1: W293 blank line contains whitespace Line 47:1: W293 blank line contains whitespace Line 52:1: W293 blank line contains whitespace Line 83:1: W293 blank line contains whitespace Line 113:1: W293 blank line contains whitespace

Line 9:121: E501 line too long (143 > 120 characters) Line 76:121: E501 line too long (133 > 120 characters)

Line 3:121: E501 line too long (124 > 120 characters) Line 27:121: E501 line too long (132 > 120 characters)

Line 14:121: E501 line too long (126 > 120 characters)

Line 8:1: F401 'fedot.core.repository.operation_tags_n_repo_enums.DataOperationTagsEnum' imported but unused

Line 9:1: E305 expected 2 blank lines after class or function definition, found 1 Line 10:121: E501 line too long (123 > 120 characters) Line 11:121: E501 line too long (127 > 120 characters) Line 12:121: E501 line too long (127 > 120 characters) Line 13:121: E501 line too long (133 > 120 characters) Line 14:121: E501 line too long (137 > 120 characters) Line 16:121: E501 line too long (127 > 120 characters) Line 20:121: E501 line too long (136 > 120 characters)

Line 13:1: F401 'fedot.core.repository.tasks.Task' imported but unused Line 15:1: E302 expected 2 blank lines, found 1 Line 45:65: F821 undefined name 'EvaluationStrategy' Line 71:1: W293 blank line contains whitespace Line 74:1: W293 blank line contains whitespace Line 75:32: E251 unexpected spaces around keyword / parameter equals Line 75:34: E251 unexpected spaces around keyword / parameter equals Line 77:1: W293 blank line contains whitespace Line 91:1: W293 blank line contains whitespace Line 95:1: W293 blank line contains whitespace Line 105:1: W293 blank line contains whitespace Line 113:1: W293 blank line contains whitespace Line 116:1: W293 blank line contains whitespace Line 120:1: W293 blank line contains whitespace Line 127:9: E117 over-indented Line 130:1: W293 blank line contains whitespace Line 174:1: W293 blank line contains whitespace Line 178:24: E127 continuation line over-indented for visual indent Line 179:24: E127 continuation line over-indented for visual indent

Line 1:1: F401 'enum.Enum' imported but unused Line 1:1: F401 'enum.IntEnum' imported but unused Line 2:1: F401 'itertools.chain' imported but unused Line 3:1: F401 'json' imported but unused Line 4:1: F401 'collections.defaultdict' imported but unused Line 5:1: F401 'dataclasses.dataclass' imported but unused Line 6:1: F401 'pathlib.Path' imported but unused Line 7:1: F401 'typing.Union' imported but unused Line 10:1: F401 'fedot.core.repository.operation_tags_n_repo_enums.ALL_TAGS' imported but unused Line 10:1: F401 'fedot.core.repository.operation_tags_n_repo_enums.DataOperationTagsEnum' imported but unused Line 10:1: F401 'fedot.core.repository.operation_tags_n_repo_enums.ModelTagsEnum' imported but unused Line 10:121: E501 line too long (136 > 120 characters) Line 16:1: F401 'fedot.core.repository.json_evaluation.import_enums_from_str' imported but unused Line 16:1: F401 'fedot.core.repository.json_evaluation.import_strategy_from_str' imported but unused Line 16:1: F401 'fedot.core.repository.json_evaluation.read_field' imported but unused Line 20:5: F401 'fedot.core.operations.evaluation.evaluation_interfaces.EvaluationStrategy' imported but unused Line 33:1: W293 blank line contains whitespace Line 40:1: W293 blank line contains whitespace Line 44:1: W293 blank line contains whitespace Line 102:1: W293 blank line contains whitespace Line 104:121: E501 line too long (125 > 120 characters)

Line 7:1: F401 'fedot.core.repository.operation_types_repo_enum.OperationReposEnum' imported but unused

Line 23:42: F821 undefined name 'Pipeline'

pep8speaks avatar Jan 19 '24 15:01 pep8speaks

Code in this pull request still contains PEP8 errors, please write the /fix-pep8 command in the comments below to create commit with automatic fixes.

Comment last updated at

github-actions[bot] avatar Jan 19 '24 15:01 github-actions[bot]

/fix-pep8

kasyanovse avatar Jan 28 '24 18:01 kasyanovse

Codecov Report

Attention: Patch coverage is 79.82196% with 68 lines in your changes are missing coverage. Please review.

Project coverage is 78.90%. Comparing base (5e726e9) to head (f049db2). Report is 7 commits behind head on master.

:exclamation: Current head f049db2 differs from pull request most recent head 8dc0dc3. Consider uploading reports for the commit 8dc0dc3 to get more accurate results

Files Patch % Lines
fedot/core/repository/operation_types_repo_enum.py 81.15% 26 Missing :warning:
...edot/core/repository/operation_types_repository.py 68.51% 17 Missing :warning:
fedot/api/fedot_cli.py 0.00% 7 Missing :warning:
fedot/api/api_utils/presets.py 86.84% 5 Missing :warning:
...ot/core/composer/gp_composer/specific_operators.py 20.00% 4 Missing :warning:
fedot/core/composer/meta_rules.py 25.00% 3 Missing :warning:
...t/api/api_utils/assumptions/assumptions_builder.py 90.00% 1 Missing :warning:
fedot/core/composer/composer_builder.py 50.00% 1 Missing :warning:
fedot/core/operations/automl.py 50.00% 1 Missing :warning:
fedot/core/operations/factory.py 88.88% 1 Missing :warning:
... and 2 more
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #1253      +/-   ##
==========================================
- Coverage   80.05%   78.90%   -1.15%     
==========================================
  Files         149      152       +3     
  Lines       10278    10316      +38     
==========================================
- Hits         8228     8140      -88     
- Misses       2050     2176     +126

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 28 '24 18:01 codecov[bot]

Closed as obsolete

nicl-nno avatar May 07 '24 14:05 nicl-nno