Dmitry

The MCTS implementation here works roughly like this (pseudocode): ```python def mcts(observation): root_predicted_value, stuff = model.initial_inference(observation) root = Node() root.expand(stuff) root.add_exploration_noise() for _ in range(num_simulations): leaf = find_unexpanded_leaf() # here...

TF v2

1

This should be rewritten. I've made this PR only to ensure that we don't forget about the existence of this branch. Fixes #417.

Port TensorFlow notebooks to TensorFlow v2

1

We are obligated to provide assignments in TensorFlow in Coursera (but we can provide PyTorch versions in addition to TF). That said, our TF notebooks are designed for TF v1...

There are a lot of things in the PyTorch notebook that make this assignment a lot easier. They should be backported to the TF version. One thing to note is...

```python device = 'cuda' if torch.cuda.is_available() else 'cpu' ... model = model.to(device) ```

Better Docker instructions

1

We have instructions for setting up local Docker at https://github.com/yandexdataschool/Practical_RL/tree/master/docker. However, they are unclear, as reported in the following threads: * https://www.coursera.org/learn/practical-rl/discussions/all/threads/E6IkT54xEemB7BKA79O1vg * https://www.coursera.org/learn/practical-rl/discussions/all/threads/urpCnVhlEeiIjg6nmV99lg Need to review problems reported in...

coursera/week6/seq2seq/basic_model_tf.py hangs with TF 1.14.0

3

https://www.coursera.org/learn/practical-rl/discussions/all/threads/b4Bm1b6OEemlhhJkLrq7mA reports that the honor track assignment hangs on Colab with the current TF version (1.14.0), but works with an old one (1.6.0). I have successfully reproduced the issue. The...

Improve week06/a2c-optional

3

via @q0o0p Закончила проверять week6 (но некоторые всё ещё досдают) В ШАД эту домашку сдали 108 человек, из них 26 сдали ActorCritic. Один прислал Кунг-Фу из week 08. В ВШЭ...