The Tutorial part should update

[ruyikang]

Excuse me! Firstly, thanks for your working. This is fantastic work. But, you may need to update the tutorial notebook. Some examples are very old. The quick-start notebook is OK, only small problem, but the tutorial notebook has small bugs due to the update of reverb. Now the datasets.make_reverb_datasets no need environment_spec and transition_adder, but it need table. Here, I use reverb 0.7.2, the suggested version in setup.py.

So, could anyone show me how to write the code of datasets for the learner?

Thanks

[ruyikang]

Because, the make_reverb_dataset no need environment_spec and transition_adder, so I just delete them. Now, the code is

dataset = datasets.make_reverb_dataset( server_address=replay_server_address, batch_size=256 )

But, when I run it, Error happened.

ValueError: Table priority_table at localhost:39847 does not have a signature.

So, how to write a dataset now?

[ruyikang]

OK, I found that the dataset has no problem, but the buffer has problem, when use reverb.Table, you do not include the signature. Then, in dataset, we will use that table and cause above bug.

So, I add signature in reverb.Table. But, another problem. The D4PG.Learner now need dataset_iterator not dataset. So, here I change dataset=dataset to dataset_iterator=dataset__iter__(). Then, the code can run. But, the performance is very bad.

Episode: 0 | Return: -9.003614 | Learner steps: 0 | Actor steps: 999 Episode: 1 | Return: -28.733247 | Learner steps: 10 | Actor steps: 1998 Episode: 2 | Return: -69.013762 | Learner steps: 20 | Actor steps: 2997 Episode: 3 | Return: -78.320042 | Learner steps: 30 | Actor steps: 3996 Episode: 4 | Return: -80.337008 | Learner steps: 40 | Actor steps: 4995 Episode: 5 | Return: -81.184628 | Learner steps: 50 | Actor steps: 5994 Episode: 6 | Return: -79.512312 | Learner steps: 60 | Actor steps: 6993 Episode: 7 | Return: -80.462558 | Learner steps: 70 | Actor steps: 7992 Episode: 8 | Return: -79.935318 | Learner steps: 80 | Actor steps: 8991 Episode: 9 | Return: -80.833340 | Learner steps: 90 | Actor steps: 9990

Is it normal? I think it has problems.

[bshahr]

Hi ruyikang,

Apologies for the late reply, we were making the final touches on the updated version of the code and paper. Please refer to the new and more simple quickstart and tutorial guides. Feel free to provide feedback and reopen the issue if the two notebooks do not work out-of-the-box.

Thanks for your interest, happy Acming!

Bobak