Question about delta ee pose control

[j-thapa]

I am pretty new in this field so it might be a simple question but it has been bugging me for a while. I am using end effector pose (x, y, z, Rx, Ry, Rz) as the action space. It seems in some action spaces choice, it violates the limit of robot so it get stuck or doesnt move at all in that action choices by the network. How can I deal with this issue? It seems hard to manually figure out this limitation to mask those action choices, maybe I should use some penalty when action choices results in no movement of the agent ?

My question might not be clear, but am willing to elaborate more on this if needed. Thanks

[StoneT2000]

You are correct. With end effector (EE) control eventually you are asking the robot to move the EE to somewhere impossible. In these cases essentially there is no solution as to what joint angles will achieve the impossible EE so in fact any action submitted that does not return the robot to some possible EE pose will do essentially nothing.

Masking the action choices might be a very complex problem and I am not sure what the solution is to masking continuous action spaces with EE controllers. I'm aware action masking (at least in discrete action spaces) is quite useful for RL use-cases, but this is often not used in continuous control.

[j-thapa]

thanks for the detailed response on this. One further question is this issue somehow circumvented in joint control as when using joint control, I seem not to face such action choices.

[StoneT2000]

delta joint control will have the same issue if you try to move joints beyond their limits. However this is easier to mask since you know what each joints limits are. the robots current qpos and limits tell you when you can mask approximately