Question - Robot arm keeps curling up after implementing reward signal.

[Shane1967]

Hi,

I have a question regarding the implementation of dense reward signal. I was able to implement the reward signal but my test results always lead to the arm curling up or trying to go back to it's initial starting position for some reason. I trained on the reach_target task and was able to reach the target but the episode does not terminate but instead the arm tries to return to its initial starting position after reaching the target. Is this a bug on RLBench or could the problem be with the algorithm on my end? Attached is my dense reward code and the test result.

def reward(self,dense) -> Union[float,None]:
    if dense == 0:
        return None
    if dense:
        
        dist_reward = self.distance_to_target()


        success, terminate = self.success()
        
        if success:
            reward = dist_reward + 1.0
        else:
            reward = dist_reward

        return reward

def distance_to_target(self):    
    x1, y1, z1 = self.agent_ee_tip.get_position()
    x2, y2, z2 = self.target.get_position()
    reward = -np.sqrt((x1-x2) ** 2 + (y1-y2) **2 + (z1-z2) ** 2)
    
    return reward

[stepjam]

Hi Shane. What action more are you using? I've trained dense-reward policies with RLBench without issue, so it's likely an issue with your algorithm/reward.

[Shane1967]

Hi James,

The action mode is the default action mode. I tried running the reward with DDPG,TRPO and Option Critic and it all ends up with the arm curling up. I'm using a reward function which was able to solve the task with 2 DOF. Mine has 6 DOF being the joints. Could this be the reason?

On Tue, Feb 15, 2022 at 12:36 AM Stephen James @.***> wrote:

Hi Shane. What action more are you using? I've trained dense-reward policies with RLBench without issue, so it's likely an issue with your algorithm/reward.

— Reply to this email directly, view it on GitHub https://github.com/stepjam/RLBench/issues/150#issuecomment-1039299313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVLCXAYTZNDG54QY5S3O45TU3EVPXANCNFSM5KRVLJHQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>