google-deepmind
Coop-mining environment bug
I have been using the cooperative mining environment and training agents to perform the task. I encountered two potential issues related to multiple agents shooting at gold:
- Limited Shot Registration:
When more than two agents attempt to shoot at gold simultaneously, only the first two agents receive rewards. Although multiple agents can execute the shooting action at the same time, only two shots are registered and counted as a mining event (registering mining event in lua code).
- Gold Persistence and Excess Rewards:
When more than two agents shoot at the gold, the gold does not disappear immediately, even though two agents receive rewards. This results in agents seemingly receiving double or triple rewards for mining the same gold.
I have recorded a video with three agents, where I display each agent's actions and rewards at each time step. In the video, action index 7 corresponds to the shooting action. Reward for iron mining grants 0.9, and gold mining grants 5.9 (after applying a -0.1 penalty per shooting action).
Video Link: https://drive.google.com/file/d/1PgsrAodoNAD5wrQrW6d9IjnGyIXkWDm7/view?usp=sharing The issues occur frequently in the last 30 seconds of the video.
I appreciate any insights on why this might happen, thank you!
I am experiencing the same issue, agents are sometimes able to exploit this bug to gain unreasonably high rewards from a single gold ore. It would be greatly appreciated if this could be addressed. Thank you!
