Docs page "Usage Guide -> Automatic hyperparameter tuning"

[ryanjulian]

e.g. ray-tune and/or hyperopt

[richardliaw]

happy to help proofread!

[ryanjulian]

thanks @richardliaw !

[nikhilxb]

@ryanjulian @richardliaw Would love to use garage with automatic hyperparameter tuning like ray-tune!

Is there a draft of this doc or an existing repo that exemplifies how this can be done?

It seems that garage uses Ray already (it creates a Ray dashboard), so I'm unsure if tuning is already built in or if I can simply pass the experiment runner function (decorated with @wrap_experiment) to ray.tune as in Ray Tune: Quickstart.

[avnishn]

Hey thanks for checking out garage!

I haven't verified that you can do this myself, but you likely can.

Can you try it and let us know what happens?

We'd love to add this doc page.

Thanks, @avnishn

[nikhilxb]

It would be helpful if someone familiar with garage could explain how Ray currently is being used.

As I referred to above, running a trainer prints "View the Ray dashboard at http://127.0.0.1:8265" and the dashboard shows several cores busy with processes. If that's the case, it may not be so straightforward to simply use ray.tune, which spins up its own distributed processing infrastructure.

[avnishn]

We use ray as the backend for our parallel sampler. See the RaySampler for more information.

You should be able to use Tune while using the RaySampler. They will attach to the same Ray server and should in theory operate fine under the same Ray server.

Did this answer your question about how ray is being used in garage?

Thanks, @avnishn

[krzentner]

To add to what Avnish said, if you don't use RaySampler (and instead use LocalSampler), then garage doesn't interact with ray at all, and everything should just work out of the box.

The main difficulty with using RaySampler with a ray server not started by it is that RaySampler shuts down ray when the experiment ends. If you want to use RaySampler in that circumstance, I would recommend copying ray_sampler.py into your own project and modifying it to not shut down ray.

Also, if you're using PyTorch on cpu, then you can use MultiprocessingSampler to sample in parallel, which also doesn't use ray.

[nikhilxb]

Thanks, that makes sense! I'll play around with it.

[nikhilxb]

One requirement of ray.tune is that you tune.report the performance metric(s) each epoch so the search algorithm can evaluate hyperparameter combinations and terminate bad performing ones early. Is there a way to hook into the training loop managed by Trainer to extract relevant performance metrics?