Zhouyou Gu

Thanks for your reply! The program ran on a virtual machine with ubuntu 20.04, CUDA 11.2, GPU 2080Ti. I use scs-python interface at [scs-python](https://github.com/bodono/scs-python), which is compiled with `--scs --gpu`....

Thx for the suggestions! I actually have tried every method you have mentioned to speed up the process :). Nevertheless, since GPU may not possibly be faster, I will just...

Hi Hanghoo, the thread (or setting asynchronization = True) is only useful in online experiments. When set to false in the offline experiment, the algorithm will behave sequentially as "generate...

Hi hanghoo, _per_w_multiplier() is a function to adjust the weight of each sample (or each transition) according to the delay of each user's queue. This is used to achieve the...

Yes, that's correct. Or, more precisely, weights in l_critic = torch.mul(l_critic_per_batch, weights) and l_actor = torch.mul(l_actor_per_batch, weights) is the correction of bias caused by importance sampling. ret_per_e = to_numpy(l_critic); ret_per_e...

Hi, hanghoo. For 1, I did not use SAC before, so I do not know about it. For 2, no difference.

Hi @xiaohui7 , yes, the offline codes only need to run that command under "controller_src". The results will be recorded in tensorboard. To see the results, open the tensorboard interference...

Hi @xiaohui7 , these pictures in the paper are generated by MATLAB using the data recorded in tensorboard. Here, tensorboard is used as a data storage system with a visualizer,...

Hi @xiaohui7 , the MATLAB codes that plot the results are not shared. You may have a look at MATLAB's document on functions "plot", "bar", "image" for the figures in...

Hi @xiaohui7 DDPG class in that file uses multi-head, which should be exactly the same as MultiheadcriticDDPG (here, I made a new class just for differentiation if I remember this...