ray icon indicating copy to clipboard operation
ray copied to clipboard

[RLLib][Air] MLFlow parsing of RLLib evaluation and custom metrics

Open olipinski opened this issue 2 years ago • 0 comments

Why are these changes needed?

Currently, RLLib metrics like the evaluation metrics and custom_metrics are not parsed as they are dicts. This checks for these two specific cases and adds them to MLFlow for tracking.

I chose these two specifically, as including all the other metrics would clutter MLFlow imo. But please tell me if you think that we should have a more general parsing.

Related issue number

N/A

Checks

  • [x] I've run scripts/format.sh to lint the changes in this PR.
  • [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
  • [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • [ ] Unit tests
    • [ ] Release tests
    • [x] This PR is not tested :(

olipinski avatar Jul 19 '22 13:07 olipinski