🔧 Fixed the Torque Input Issue for Torque Mode

[behradkhadem]

I surfed the web and turns out the link you sent in https://github.com/maxspahn/gym_envs_urdf/issues/230#issuecomment-1708340979 had the correct solution. Now robots (I tested point and panda) react to torque input, but it seems that (at least in the case of panda arm) robot falls. By falling I mean that robot can't preserve the initial posture and the links kinda fall to the ground. It reacts to the input torque but with a disfigured position. Is it natural? We don't have this behavior in velocity mode.

I'd be happy if you test it.

[maxspahn]

hi @behradkhadem ,

great that you investigated that. If the robot is falling, that is because there is no gravity compensation. That makes totally sense. We could try to turn off gravity, but then obstacles also don't fall down.

The other thing, we can move the line you added to a more central place in the reset, so that it is not called in every time step.

[maxspahn]

By the way, there is already a method for disable the velocity control, like you did it. It is called disable_velocity_control.

[maxspahn]

Maybe we can solve #15 in this PR as well.

[behradkhadem]

We could try to turn off gravity

I think this is a bad idea. In that pick and place becomes impossible.

By the way, there is already a method for disable the velocity control, like you did it. It is called disable_velocity_control.

My bad! I didn't see it, sorry.

The other thing, we can move the line you added to a more central place in the reset, so that it is not called in every time step.

Would you elaborate on this matter please?

[behradkhadem]

Maybe we can solve #15 in this PR as well.

You propose that we have a singular friction for all joints (like the one you've implemented self._friction = 0.1) or we let user input an array of frictions for all joints (like this: [0.1, 0, 0, 0, 0.2, 0.05])?

[maxspahn]

Would you elaborate on this matter please?

Just use the function disable_velocity_control, so that we don't have to loop through the joints again.

You propose that we have a singular friction for all joints (like the one you've implemented self._friction = 0.1) or we let user input an array of frictions for all joints (like this: [0.1, 0, 0, 0, 0.2, 0.05])?

Could you use a default float value 0.1, and pass an optional argument somewhere, probably through a dedicated function set_friction where the friction can be set by a single float value or an array of values.

I think this is a bad idea. In that pick and place becomes impossible.

What is the alternative though?

[behradkhadem]

Could you use a default float value 0.1, and pass an optional argument somewhere, probably through a dedicated function set_friction where the friction can be set by a single float value or an array of values.

I'll work on it.

What is the alternative though?

I'm not in the position to suggest anything. Gravity compensation is a special and complicated matter for itself and there are papers that argue its importance in RL (like this for example). So, I don't know what to do. What is your opinion?

PS: Sorry for the delay in answering. Over the past few weeks, I've been dealing with serious health issues and wasn't able to focus on the project. I hope that I be able to cooperate on the project and make it better from now on.

Thanks.

[maxspahn]

I'm not in the position to suggest anything. Gravity compensation is a special and complicated matter for itself and there are papers that argue its importance in RL (like this for example). So, I don't know what to do. What is your opinion?

According to the paper, it is important that the RL agent should not learn the gravity compensation part. Could you try to use the inverse dynamics only for the gravity part and then add the action-torque to it?

[behradkhadem]

According to the paper, it is important that the RL agent should not learn the gravity compensation part. Could you try to use the inverse dynamics only for the gravity part and then add the action-torque to it?

I asked a few of my professors about this in past week and they either said they didn't know, or they said that in real life robots, also, we should deal with gravity compensation. So, the agent itself should learn to compensate the gravity. Although we can implement the compensator separately for simple robots, in complicated ones we will face challenges. So, to my limited understanding so far, it's better if the agent learns the gravity compensation part alongside with pick-and-place stuff. Apparently learning the compensation part can be facilitated with minimizing the energy used by each motor in a robot or penalizing the robot for using excessive torque. In which I'm not sure whether is it possible with data we have in our observation dict.

But the robot now responds to understandable amount of torque, correctly and we're good in that regard.

[behradkhadem]

I also added the friction torque input to the GenericRobot constructor (addressing this: https://github.com/maxspahn/gym_envs_urdf/issues/15). I tried adding it to the UrdfEnv but since we have a list of GenericRobot in each UrdfEnv, I thought it would be better if we add friction data of each robot inside the robot constructor itself. But adding the torque input to UrdfEnv is totally doable, we'll just need the robot id for each input.

Is there anything I forgot to add?