BigDL-2.x
BigDL-2.x copied to clipboard
Bigdl2.0 migration plan for ray/automl/orca.automl/chronos
Overall plan
The migration plan for ray/automl is
-
ray
will be put intoorca
. -
automl
will be integrate withorca.automl
(part withchronos
).
And the pip install dependency for chronos
will be orca
only.
Therefore, the overall migration plan could be divided into two steps to reduce error and avoid confliction, the first step is to clean up the code structure for orca.automl
and chronos
, note that automl
will be removed. And the second step is to migrate zoo.ray
, zoo.orca.automl
and zoo.chronos
to BigDL2.0.
(Ordered) task list
Step 1: clean up code structure on zoo master
Automl
Step 1: Code structure in automl
Move to chronos
- [x] zoo/automl/pipeline/base -> zoo/chronos/pipeline #4633
- [x] zoo/automl/regression/base -> zoo/chronos/regression #4633
- [x] zoo/automl/recipe/base -> zoo/chronos/config #4633
Internal clean up
- [x] zoo/automl/common/metrics -> zoo/automl/metrics #4636
- [x] zoo/automl/search/ray_tune_search_engine -> zoo/automl/search/ray_tune #4648
- [x] zoo/automl/logger/tensorboardxlogger.py -> zoo/automl/search #4650
- [x] zoo/automl/common/utils -> zoo/automl/search,
utils.py
needs refactor #4641 - [x] zoo/automl/common/parameters -> zoo/automl/search #4642
Step 2: Move to orca.automl
- [x] zoo/automl/metrics -> zoo/orca/automl #4661
- [x] zoo/automl/model -> zoo/orca/automl #4667
- [x] zoo/automl/search -> zoo/orca/automl #4670
- [x] remove zoo/automl #4680
Chronos
- [x] chronos/model/forecast -> chronos/forecaster #4651
- [x] chronos/model/anomaly -> chronos/anomaly #4669
Below tasks are part of AutoTSTrainer
deprecation (issue #4675)
- [ ] add deprecated warning in code/docs for
AutoTSTrainer
andTSPipeline
#4786 - [ ] chronos/autots/experimental -> chronos/autots #4786
- [ ] chronos/autots/forecast.py -> chronos/autots/deprecated #4786
- [ ] chronos/pipeline -> chronos/autots/deprecated #4786
- [ ] chronos/regression -> chronos/autots/deprecated #4786
- [ ] chronos/config -> chronos/autots/deprecated #4786
- [ ] chronos/feature -> chronos/autots/deprecated #4786
- [ ] chronos/preprocessing -> chronos/autots/deprecated #4786
Step 2: Migrate to BigDL 2.0
We will split into small features and migrated to BigDL2.0 separately.
- [x] zoo/ray -> bigdl/orca/ray
- [x] zoo/orca/automl -> bigdl/orca/automl
- [ ] zoo.chronos -> bigdl.chronos (needs to wait till
orca
has been migrated)
Note that we will not keep git history when migrating to BigDL2.0
.
The code structure for Chronos
will be
- autots
- model
- autotsestimator.py
- tspipeline.py
- deprecated
- forecast.py
- pipeline
- regression
- config
- feature
- preprocessing
- forecast
- anomaly
- data
- model (time series models in chronos.model)
- examples
- use_case
The code structure for orca.automl
will be
- xgboost
- model
- base_pytorch_model.py
- pytorch_utils.py
- base_keras_model.py
- search
- ray_tune
- ray_tune_search_engine.py
- utils.py
- tensorboardlogger.py
- parameters.py
- utils.py
- base.py
- ray_tune
- auto_estimator.py
- hp.py
- metrics.py
And chronos.autots
will depend on orca.automl
only.
For chronos:
- if we deprecate autots/forecast.py and related files, does it mean we remove the old AutoTSTrainer as a whole? Are we confident to let users switch to the new AutoTSEstimator in the next release? @TheaperDeng
- anomaly detectors does not have wrappers as Forecasters, so we don't have an folder for anomaly now. It is okay to move chronos/model/anomaly to chronos/anomaly. Later we may add change point detection and we can create a new folder.
- autotsestimator currently only works for forecasters. That's why we put it in package forecast.autotsestimator in the past. It is okay for me we just move it to the package autots.autotsestimator. Later we may enable autotsesitmator to tune other models besides forecasters.
For chronos:
- if we deprecate autots/forecast.py and related files, does it mean we remove the old AutoTSTrainer as a whole? Are we confident to let users switch to the new AutoTSEstimator in the next release? @TheaperDeng
- anomaly detectors does not have wrappers as Forecasters, so we don't have an folder for anomaly now. It is okay to move chronos/model/anomaly to chronos/anomaly. Later we may add change point detection and we can create a new folder.
- autotsestimator currently only works for forecasters. That's why we put it in package forecast.autotsestimator in the past. It is okay we just move it to the package autots.autotsestimator.
AutoTSEstimator
and TSDataset
can handle the features provided by AutoTSTrainer
either in the same/a better way. The autotuning of MTNet is the only feature left behind and we will support it when it is requested.
AutoTSTrainer
will also be deprecated in API doc, user guide and quick start this week.
For chronos:
- if we deprecate autots/forecast.py and related files, does it mean we remove the old AutoTSTrainer as a whole? Are we confident to let users switch to the new AutoTSEstimator in the next release? @TheaperDeng
- anomaly detectors does not have wrappers as Forecasters, so we don't have an folder for anomaly now. It is okay to move chronos/model/anomaly to chronos/anomaly. Later we may add change point detection and we can create a new folder.
- autotsestimator currently only works for forecasters. That's why we put it in package forecast.autotsestimator in the past. It is okay for me we just move it to the package autots.autotsestimator. Later we may enable autotsesitmator to tune other models besides forecasters.
chronos/model/anomaly has been moved to chronos/anomaly (PR #4669)
I will work with junwei to deprecate AutoTSTrainer
and move AutoTSEstimator
, TSPipepline
out of experimental
. (Issue #4675)
We can have a few modifications on the first-level folders. The major entrance of chronos will be 5
- chronos/forecaster - for forecasting
- chronos/detector - for anomaly, changepoint, seasonality dections
- chronos/detector/anomaly
- chronos/detector/changepoint
- chronos/simulator - for time series generation and simulations
- chronos/data - for data processing, i.e. TSDataset
- chronos/autots - automated training
Task list for the modified chronos structure.
- [x] chronos/forecast -> chronos/forecaster
- [x] chronos/anomlay -> chronos/detector/anomaly