reimplementation performance

[xiaojun-chang]

Hi,

Thanks for sharing the code. I strictly follow the instruction of this repo, but cannot get the performance reported. I can only get the following performance for SAVN.

{ "GreaterThan/1/spl": 0.13895125447326895, "GreaterThan/1/success": 0.384, "GreaterThan/5/spl": 0.11553156460140891, "GreaterThan/5/success": 0.25898203592814373, "done_count": 0.974, "ep_length": 27.018, "spl": 0.13895125447326895, "success": 0.384, "total_reward": 1.6536600000000015, "total_time": 0.5675430691242218 }

Any suggestion? Thanks.

Xiaojun

[mitchellnw]

To confirm you're using the same version of pytorch etc. ?

And you're running on the val set to choose the best model then running that on the test set?

If yes then I'm sorry -- I don't know why that would be. (If your intention is research) I would recommend reporting these numbers instead of ours, and just detail that you ran the code from the work.