Andriy Drozdyuk

Will this work? In `application.ex`: ``` worker(DogStatsd, [%{:host => "localhost", :port => 8125}, [{:name, :dogstatsd}]]) ``` Then in code ``` require DogStatsd ... DogStatsd.increment(:dogstatsd, "page.views") ```

So I guess replay buffer doesn't only store "transitions", but "sequences", and a transition is a special case of such a sequence?

Oh got it. Perhaps adding dimensions to the docs is a good idea. Looking at that picture above it is not that clear. But after your explanation it is immediately...

Probably all the things that env step returns? reward, state, info? Initially I expected to just have a pointer to the replay buffer, but then I realized that it's the...

Learning metrics?

This is very complicated, I don't want to mess up stuff. For example, collector itself doesn't return the info objects: https://github.com/thu-ml/tianshou/blob/fc251ab0b85bf3f0de7b24c1c553cb0ec938a9ee/tianshou/data/collector.py#L317 It's also not clear to me how `result["rews"]` ([here](https://github.com/thu-ml/tianshou/blob/22d7bf38c8cad70be89e44e91666dcc90e193730/tianshou/trainer/onpolicy.py#L121))...

Here is an example of where it might be useful: https://github.com/openai/safety-gym#getting-started Stop when costs are below some threshold. ``` >>> info {'cost_hazards': 0.0, 'cost': 0.0} ``` I'm happy if only...

I'm not sure (but I'm not against doing the work), just bringing it up for discussion to see what your thoughts are. But I know personally for me a line...

Found an example of possible integration here: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/domain_templates/reinforce_learn_Qnet.py But it just seems to remove all the nice tianshou concepts and replace them with pytorch ones - and I'm not sure...

Resetting works on ur10. But action is broken - doesn't do anything. Any tips?