crab icon indicating copy to clipboard operation
crab copied to clipboard

[Feature Request] Decouple `Benchmark` and `MultiEnvironment`

Open dandansamax opened this issue 6 months ago • 0 comments

Required prerequisites

  • [X] I have searched the Issue Tracker that this hasn't already been reported. (+1 or comment there if it has.)

Motivation

Current Benchmark class in coupled with GraphEvaluator and Multiple Environment features. While uses may want to use the environments only without evaluation.

Solution

The Benchmark should be divided into two classs:

  • Benchmark class is reponsible for organizing Tasks and Evaluators
  • MultiEnvironment class is responsible for manage agent intreaction with multiple environments

Users can use MultiEnvironment class with agents to operate environments without evaluators.

Additional context

No response

dandansamax avatar Aug 25 '24 11:08 dandansamax