skrl icon indicating copy to clipboard operation
skrl copied to clipboard

Fix TD3 DDPG Implementation: Move Sampling Inside Gradient Step Loop

Open alessandroassirelli98 opened this issue 1 year ago • 0 comments

This pull request addresses a discrepancy between the original TD3 and DDPG paper's algorithm and the current implementation in the repository. Specifically, the original implementation performs the sampling step outside of the gradient step loop, which diverges from the methodology outlined in the paper. We have corrected this by moving the sampling process inside the gradient step loop, aligning the implementation more closely with the intended algorithmic procedure described in the original paper and SpinningUp description.

alessandroassirelli98 avatar Apr 03 '24 09:04 alessandroassirelli98