agential icon indicating copy to clipboard operation
agential copied to clipboard

Published 20 hours ago • verified publisher icon agential-ai

[Feature Request]: CRITIC

Open alckasoc opened this issue 10 months ago • 0 comments

Implement:

  • [x] HotpotQA
  • [x] TriviaQA
  • [x] #87
  • [x] #71
  • [x] #73
  • [x] #72
  • [x] #80
  • [x] #81
  • [x] #83
  • [ ] #84
  • [ ] #85
  • [ ] #86 (includes ALFWorld & WebShop)

The decision-making benchmarks (ALFWorld, WebShop, and AgentBench) will require more design work. Swapping out the prompts won't suffice.

Run:

  • [ ] HotpotQA
  • [ ] TriviaQA
  • [ ] AmbigNQ
  • [ ] GSM8k
  • [ ] SVAMP
  • [ ] TabMWP
  • [ ] MBPP
  • [ ] HumanEval
  • [ ] ALFWorld
  • [ ] WebShop
  • [ ] AgentBench (includes ALFWorld & WebShop)

alckasoc avatar Apr 21 '24 07:04 alckasoc