feat (environment): Swarm Covering Environment

[zombie-einstein]

I was looking at implementing an environment similar to this one from petting-zoo (which itself I believe is taken from an older OpenAI multi-agent env library) where a team of agents have to simultaneously cover (i.e. come within a certain range of) a set of targets (of the same size) whilst not colliding.

It should be possible to reuse some of the existing code from the search-and-rescue environment such as the agent local visualisation and matching agents to targets (the main difference between this and search-and-rescue is that the targets are fully visible and need to be continually covered instead of only found once).

• State: A 2d space with a set of targets distributed across the space, and a set of agents with position updated each step according to their velocity. Targets can be in an uncovered/covered state if an agent is within a fixed range of them.
• Rewards: In the original implementation they use a distance based rewards between agents and targets, but there's a couple other options that could be considered:
  • Shared rewards based on the fraction of covered targets
  • Individual rewards when an agent covers a target, that are divided if agent are covering the same target.
  • Sparse rewards awarded when the agents succeed in covering all the targets.
• Actions: Agents individually update their velocity. In contrast to the search-and-rescue I was looking into a simplified drone/quadcopter type flight model, i.e. agent pitch/roll/rotate with some momentum (but in 2d space).

[sash-a]

Hey @zombie-einstein this would be great to have and is a classic cooperative MARL env, but would be great if we could make it harder and put our own spin on it as it already exists in JaxMARL.

Also Jumanji has the tricky requirement of needing to put an industry related spin on the environments

[zombie-einstein]

Hi @sash-a, I was thinking that the use of a more realistic drone-like flight model, reusing the local vision model from search-and -rescue environment, and increasing the scale of the number of agents would add some more difficulty/realism?

I was thinking it could be interesting to have the task be having the agents maximise the coverage of an area/shape, but then I was not totally sure there was an efficient algorithm to derive how much area of a shape is covered by another collection of circles (past a couple collection of 1-2 circles), but if you have an idea on this could be a nice extension.

In this case is it could be something along the lines of a drone surveying/observation environment where the agents are most effectively trying to cover/observe/guard an area.

[sash-a]

Interesting, I guess it could be squares instead of circles and then it should be much easier to work out the total area covered.

This env also scales nicely as you increase the number of agents it becomes harder to cooperate.

I think the drone observation is a good "realistic" spin on the problem

[zombie-einstein]

I figured circles are easy enough to get the overlap with, I was picturing a larger circular area that the agents need to cover (rather than individual targets) with their own smaller circular view regions, but yeah could not quite settle on the the overlapping area between large number of agents.

I could knock up a prototype, a lot of the code for this would overlap with the search-and-rescue environment so should hopefully be relatively straightforward (though some of my time estimates can be off 😂).

[sash-a]

Sounds good @zombie-einstein please do try make a prototype and we can take it from there