BigDL-2.x icon indicating copy to clipboard operation
BigDL-2.x copied to clipboard

Chronos: Move `orca.automl.metrics` to `Chronos`

Open TheaperDeng opened this issue 3 years ago • 1 comments

  • Since orca.automl is used for customized model for any area, while most of the metrics in orca.automl.metrics is designed and optimized for time series tasks.
  • The built-in metrics in orca.automl can be a simple map between string and sklearn.metrics
  • This will help Chronos to be self-contained.

TheaperDeng avatar Oct 18 '21 01:10 TheaperDeng

Since in ·orca.automl·, AutoXGBoost also uses metrics, it might not be convenient to directly move metrics from orca to chronos.Otherwise, orca need depend on chronos.

For chronos, in the cases of

  1. Single node acceleration with nano: we could directly use TorchMetrics in pytorch-lightning, which might also be more efficient regarding both performance and memory.
  2. Distributed training or distributed tuning, both of which will depend on orca and could choose to use metrics in orca.automl.

[TBD]However, we might move metrics to nano, since it is for single node only...

shanyu-sys avatar Oct 19 '21 04:10 shanyu-sys