DSAC-v1
DSAC-v1 copied to clipboard
Jingliang-Duan
No results are output
It seems always rerun these codes.
while self.experience_out_queue[index].empty() and not self.stop_sign.value:
index = np.random.randint(0, self.args.num_buffers)
time.sleep(0.1)
Hi
Did you install Mujoco? This code in running in multi processes. In ubuntu system, some people’s python/pytorch environment does not support multi-process, so you should first test whether the network in ACTOR.py successfully feeds forward once. This is a very strange bug, it may run normally in some ubuntu systems but fail in some other ubuntu systems. But it always works well in Windows and Macos. Therefore, I suggest you first test whether the actor process is running normally.
Best wishes,
Jingliang
On Aug 26, 2020, at 10:59 PM, 徐天齐 [email protected] wrote:
I ran this project, but there is nothing output. Please tell me why?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Jingliang-Duan/Distributional-Soft-Actor-Critic/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOEC2WU2JK6QPGZWP4OCIL3SCUPMNANCNFSM4QL5SKIQ.
I found ACTOR.py put data in "experience_in_queue". But Leaner.py gets data from "experience_out_queue, so the learner's buffer queue is always empty. There may be a lack of code to transfer data.
How many buffur and learner processes do you use? "parser.add_argument('--initial_buffer_size', type=int, default=2000)” this parameter means that the buffer.py will not put data in Experience_out_queue until the buffer size is greater than 2000. Is it possible that your actor.py is too slow to collect data from Environment？
Experience_in and Experience_out are two different queues, I think the speed of buffer.py putting data in Experience_out should be faster than learner.py getting data from this queue if the number of learner processes and buffer processes are moderate.
This code is used by many people around me. I think it will work. Please feel free to contact me if you have any other questions
On Aug 27, 2020, at 10:24 AM, 徐天齐 [email protected] wrote:
I found ACTOR.py put data in "experience_in_queue". But Leaner.py gets data from "experience_out_queue, so the learner's buffer queue is always empty. There may be a lack of code to transfer data.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Jingliang-Duan/Distributional-Soft-Actor-Critic/issues/1#issuecomment-681304115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOEC2WWI2FQZROGSNN4SODLSCW7VHANCNFSM4QL5SKIQ.
I found out what the problem was. It cannot work when using multiprocessing actor, the actor.py run() stops at "get_action()"
for i in range(self.args.max_step-1):
state_tensor = torch.FloatTensor(self.state.copy()).to(self.device)
if self.args.NN_type == "CNN":
state_tensor = state_tensor.permute(2, 0, 1)
self.u, _, _ = self.actor.get_action(state_tensor.unsqueeze(0), False)
self.u = self.u.squeeze(0)
in get_action() fuction of Model.py, it stops at "forward()"
def get_action(self, state, deterministic, epsilon=1e-4):
mean, log_std = self.forward(state)
Ｍoreover, I run the program on ubuntu 16.04, torch==1.6.0
@Termset I am also facing similar problem. When I run Actor.py, I get following response
khan@khan:~/rl_ws/Distributional-Soft-Actor-Critic$ python3 Actor.py
!!!!!!
!!!!!!
!!!!!!
!!!!!!
!!!!!!
which means it is working, I guess. Even then when I run Main.py, I get nothing. Could you please guide me? Thank you for your help.
@Termset @abdul-mannan-khan
Hi, sorry for the late response. As @Termset said, the code stop at forward() of the NN module due to the Multiprocessing. For the same environment configuration, this problem will only happen in the ubuntu system, it works well in WIndows or MacOS. My friend has encountered the same problem in ubuntu before. We used the same version of ubuntu and torch, but the code only works for me. We have not identified a specific reason for the weird problem. I guess ubuntu requires some special configuration for running multiprocess within torch. He finally solved this problem by directly copying my environment. Sometimes, trying another computer also helps.
Hi, I tried to run the code but faced the same issue. For MacOs, the code requires gcc@9 which is not supported by M1. For Linux, the same issue is occurring for me too as mentioned in the issue. Can you share the environment that you used to run it on Ubuntu? @Jingliang-Duan
Hi, sorry for the late reply. Could you please try ubuntu 20.04，pytorch1. 8. 1，python3. 8. Please feel free to contact my if you have further problems. Best.
On 1/30/2022 18:42，Nitish @.***> wrote：
Hi, I tried to run the code but faced the same issue. For MacOs, the code requires @.*** which is not supported by M1. For Linux, the same issue is occurring for me too as mentioned in the issue. Can you share the environment that you used to run it on Ubuntu?
— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>
Hi, I tried to run the code but faced the same issue. For MacOs, the code requires gcc@9 which is not supported by M1. For Linux, the same issue is occurring for me too as mentioned in the issue. Can you share the environment that you used to run it on Ubuntu? @Jingliang-Duan
Dear friend, the running problem in ubuntu has be solved.
@Termset I am also facing similar problem. When I run
Actor.py, I get following responsekhan@khan:~/rl_ws/Distributional-Soft-Actor-Critic$ python3 Actor.py !!!!!! !!!!!! !!!!!! !!!!!! !!!!!!which means it is working, I guess. Even then when I run
Main.py, I get nothing. Could you please guide me? Thank you for your help.
Dear friend, the running problem in ubuntu has been solved.
I found out what the problem was. It cannot work when using multiprocessing actor, the actor.py run() stops at "get_action()"
for i in range(self.args.max_step-1): state_tensor = torch.FloatTensor(self.state.copy()).to(self.device) if self.args.NN_type == "CNN": state_tensor = state_tensor.permute(2, 0, 1) self.u, _, _ = self.actor.get_action(state_tensor.unsqueeze(0), False) self.u = self.u.squeeze(0)in get_action() fuction of Model.py, it stops at "forward()"
def get_action(self, state, deterministic, epsilon=1e-4): mean, log_std = self.forward(state)Ｍoreover, I run the program on ubuntu 16.04, torch==1.6.0
I ran this project, but there is nothing output. Please tell me why?
I ran this project, but there is nothing output. Please tell me why?
Dear friend, the running problem in ubuntu has been solved.