Hierarchical Imitation Learning

[Dhanushvarma]

Is there any existing implementation of "Hierarchical Imitation Learning" with tianshou, if not, does the library provide support to implement this algorithm.

[MischaPanch]

Hi. This is not on the current roadmap, but if you are interested in working on an implementation, I'm happy to discuss it with you.

Generally, the core team is currently more focused on improving interfaces and design than on including new algos. External contributions of new algos are welcome though!

[Dhanushvarma]

I would be interested on working on the implementation, I'll have to initially sketch out the tianshou repo, as I am not very familiar with it. It would be great if you could guide on the best way to implement the aforementioned algorithm in this framework :)

[MischaPanch]

Hi, sorry for the late answer. Great, I'll be happy to review your work and assist with the implementation. A good start is with the tutorials and the example scripts. You can have a look at the implementation of ImitationLearning.

A new algorithm is added in the steps:

1. A new policy, inheriting from BasePolicy or one of its subclasses
2. A training script using low-level interfaces. See the existing examples
3. Include the policy in the high-level Interfaces and prepare an example script

Step 3. can happen later, in a separate PR. I'm not very familiar with hierarchical imitation learning, but once you have a POC implementation, it will be a good basis for discussions. When the policy is finished, you can likely train it with the OfflineTrainer

[MischaPanch]

You can discover all existing algorithms by looking at the implementations of BasePolicy