crab
crab copied to clipboard
[Feature Request] Decouple `Benchmark` and `MultiEnvironment`
Required prerequisites
- [X] I have searched the Issue Tracker that this hasn't already been reported. (+1 or comment there if it has.)
Motivation
Current Benchmark
class in coupled with GraphEvaluator
and Multiple Environment
features. While uses may want to use the environments only without evaluation.
Solution
The Benchmark
should be divided into two classs:
-
Benchmark
class is reponsible for organizing Tasks and Evaluators -
MultiEnvironment
class is responsible for manage agent intreaction with multiple environments
Users can use MultiEnvironment
class with agents to operate environments without evaluators.
Additional context
No response