ConvLab-2 icon indicating copy to clipboard operation
ConvLab-2 copied to clipboard

[BUG] DQN training not resulting in good results on task simulator

Open kh522 opened this issue 4 years ago • 2 comments

Describe the bug I don't seem to be able to get the dqn code to be working properly. Training DQN with default configurations doesn't seem to result in good performance when evaluating on the task simulator. I seem to only be able to achieve a 1% task success rate when evaluating on the simulator.

To Reproduce Steps to reproduce the behavior:

  1. Run train.py file in dqn folder
  2. Run evaluation.py

Expected behavior Is there an expected task success rate for DQN?

kh522 avatar Oct 30 '20 20:10 kh522

Currently, we got a similar result. Still working on this.

zqwerty avatar Nov 02 '20 07:11 zqwerty

I got exactly the same issue. In addition, I can't download the pretrained model by the url in dqn.py, that is, https://convlab.blob.core.windows.net/convlab-2/dqn_policy_multiwoz.zip

ghost avatar Nov 21 '20 13:11 ghost