FAR-HO icon indicating copy to clipboard operation
FAR-HO copied to clipboard

Published 20 hours ago • verified publisher icon lucfra

pytorch support

Open ifangcheng opened this issue 5 years ago • 4 comments

This is a nice package for HPO, but it seems that it is built only on TF. Just wondering is there any version that supports pytorch? I think that will make the package more popular for HPO reseachers!

ifangcheng avatar Mar 07 '19 14:03 ifangcheng

Thanks @ifangcheng !

In fact, there is a person that's working on it; but I don't know when it will be ready. I presume that at least some part of this package (the reverse mode hypergradient) should be easier to implement in Pytorch since - as far as I know - Pytorch should support by default dynamic computational graphs. Unfortunately I don't have much experience with Pytorch myself.... but, of course, if you want to help and get involved, you'd be very welcomed! :-)

Best, Luca

lucfra avatar Mar 10 '19 21:03 lucfra

Hi,

Do you have any news on the work for the PyTorch implementation ? I'm really interested because i'd like to use specifically the reverse mode HG in PyTorch ! :D Thanks for your great work, it's still very helpful even only on TF !

Best, Antoine

AntoineHX avatar Sep 30 '19 17:09 AntoineHX

Hi @AntoineHX

We are shortly releasing a PyTorch implementation of an algorithm related to RTHO (real time hyperparamter optimization) for optimizing learning rate schedules online. Unfortunately at the moment we have no specific short term plans of releasing a general implementation/wrappers for reverse mode HG in PyTorch, but this may change in the future. I'll keep you updated!

Cheers, Luca

lucfra avatar Oct 16 '19 10:10 lucfra

Hi all,

just wanted to post the link of a new package for gradient-based hyperparameter tuning in PyTorch.

https://github.com/awslabs/adatune

Right now it only contains the code for running an enhanced version of RTHO, but it may grow in the future months.

Cheers, Luca

lucfra avatar Nov 12 '19 18:11 lucfra