Shibi He

Hi guys, I have released the code at https://github.com/ShibiHe/Q-Optimality-Tightening. Please have a look. Best, Shibi

You are asking really incisive questions. Thank you very much, Sudeep. The first question is about KNN algorithm. Due to the high dimension of the states and the dynamic LRU...

Wow, thank you very much. I am having a vacation now and will be back school in two days. I was about to tell you that I got the reply...

Hello. I checked your code and you were using KD tree. I have pushed my new commitment in which I implemented approximate KNN. Please have a look if possible. Thanks...

Hi, please refer to Explain part in the readme for details.

It is probably because when the epsilon-greedy is less likely to be greedy, the inference takes more time. I have also noticed different game behaves really different. Some games' speeds...

I think I run a few random seeds and compute an average.

I remember I wrote another code specially for doing 30 no-op test. This code base is quite old, so I do not remember the details. My new implementation was using...

Good question. Theoretically, we should only use the upper bounds after the Q is sufficiently trained and we find upper bounds stabilize the training. In practice, we just use the...

How's your improvement? I would like to make grayscaling and frame skip too.