FLAML icon indicating copy to clipboard operation
FLAML copied to clipboard
verified publisher icon microsoft

Failure cases for autogen agents

Open sonichi opened this issue 2 years ago • 2 comments

  1. extract all telescopes that are up for auction on https://astromart.com/ (using a wrong url; using wrong assumption of the html structure)
  2. Find $a + b + c$, given that $x+y \neq -1$ and \begin{align} ax + by + c & = x + 7,\ a + bx + cy & = 2x + 6y,\ ay + b + cx & = 4x + y. \end{align}. (using numpy; need the hint: use sympy)
  3. What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain? (using outdated ticker FB; need the hint: FB -> META)

sonichi avatar Jul 07 '23 21:07 sonichi

Is it possible to summarize the observed reasons for the failures?

qingyun-wu avatar Jul 11 '23 01:07 qingyun-wu

Is it possible to summarize the observed reasons for the failures?

One can run these examples and observe the chat. Then, one can try adding hints to make it work. It'll be nice to reflect from that and make it smarter in future. Would you like to try this with the learning agent?

sonichi avatar Jul 11 '23 04:07 sonichi