Interface for Environment and Control

[skim0119]

Issue by skim0119 Sunday Dec 12, 2021 at 19:05 GMT Originally opened as https://github.com/GazzolaLab/elastica-python/issues/86

---

From @tp5uiuc

I'm porting only the Environment and Controller interfaces here, as the rest are already in-place. An additional reason is that its not quite clear what needs to go into the Environment (slender-body theory is one, friction is another...), and what the interface for a Controller is going to be.

This is a WIP, so feel free to propose any changes as you see fit. Things yet to do:

• [ ] Environment mixin interface : add environment effects (fluids, friction etc, needs PR from some other branch, I'll only provide the interface)
• [ ] tests forEnvironment
• [ ] Controller mixin interface : interface for attaching arbitrary controllers (once again needs a different PR)
• [ ] tests forController

---

It'll be nice to eventually have a time-varying environment, itself governed by an independent system (say, governed by a PDE). The example I have in mind is a chemical source placed at some position in the free-space—it releases chemical concentration based on some simple diffusion law. We can make our arms "sense" this concentration.

[skim0119]

Comment by tp5uiuc Sunday Dec 12, 2021 at 19:10 GMT

---

It would be good if figure out the Controller interface here. For the Environments, you can take a look at Elastica++, we have a collision-detection system that handles friction for example.

[mstoelzle]

Would the Controller mixin mentioned here have access to the whole simulator state (e.g. not just the state of a single rod)? This is a requirement we have for the use-case I am currently working on and I am not quite sure how to implement my controller, as the NoForces classes in external_forces.py only has access to one system / rod at a time.

Also, are you still planning to implement this feature? If yes, what time frame are you expecting?

[mstoelzle]

• [ ] Controller mixin interface : interface for attaching arbitrary controllers (once again needs a different PR)

I will make a first proposal / attempt for the Controller interface (assuming that I understand its purpose correctly):

• The user can define a CustomController class:

class CustomController(BaseController):
    def __init__(self, systems: Dict[str, SystemType], step_skip: int, *args, **kwargs):
        super().__init__(systems=systems, step_skip=step_skip, *args, **kwargs)

    def apply_forces(self, systems: Dict[str, SystemType], time: float):
        for system_name, system in systems.items():
            system.external_forces += np.ones(shape=system.external_forces.shape)

    def apply_torques(self, systems: Dict[str, SystemType], time: float):
        for system_name, system in systems.items():
            system.external_torques += np.ones(shape=system.external_torques.shape)

where systems is a named dictionary of systems, which are required for running the specified controller and need to be specified to the class at initialization:

simulator.control(CustomController, systems={"rod1": rod1, "rod2": rod2, "cylinder1": cylinder1}, step_skip=1000)

The controller can be run at a lower frequency (e.g. bigger dt) than the simulation, which is determined by step_skip analogue to CallBackBaseClass.

[armantekinalp]

Hi @mstoelzle

Currently we don't have man power to pursue this issue, but it is something we wanted to have in PyElastica in future.

We faced the same issues before some other project, where we were coupling PyElastica with stable-baseline RL algorithms. For that project we followed OpenAIGym environment structure and called the RL algorithm after some PyElastica steps. If you would like to see the example here is the link.

Method you suggested can also be included in PyElastica as an other mixin method to the simulator. I think it can follow the same strategy as Forcing class but simulator can return systems that controller acts on instead of a single system.

[mstoelzle]

@armantekinalp Thank you for your answer and for your referral to the RL project. In case I decide to implement this functionality, I will try to merge it back to PyElastica as another mixin as you proposed.

Small correction for the desired control syntax to get it more in line with the usual PyElastica syntax:

simulator.control(
        systems={"rod1": rod1, "rod2": rod2, "cylinder1": cylinder1}
).using(CustomController, step_skip=1000)

[armantekinalp]

@mstoelzle sounds good. Let me know if you need anything.