Add SQL agent and Spider environment

[byronxu99]

Why are these changes needed?

Related issue number

Checks

• [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
• [ ] I've made sure all auto checks have passed.

[rickyloynd-microsoft]

Hi @byronxu99 Can you provide details about this SQL agent and Spider environment, as well as their benefits? The test dir is generally reserved for test scripts to be run by pytest or by users. New agents generally belong under agentchat/contrib.

[zwd-ms]

Hi @byronxu99 Can you provide details about this SQL agent and Spider environment, as well as their benefits? The test dir is generally reserved for test scripts to be run by pytest or by users. New agents generally belong under agentchat/contrib.

Hi @rickyloynd-microsoft I moved the agent into the notebook directory as an example, hopefully that's a better place.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (fd0094c) 32.48% compared to head (5274ade) 32.48%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1218   +/-   ##
=======================================
  Coverage   32.48%   32.48%           
=======================================
  Files          41       41           
  Lines        4907     4907           
  Branches     1120     1120           
=======================================
  Hits         1594     1594           
  Misses       3187     3187           
  Partials      126      126           

Flag	Coverage Δ
unittests	32.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rickyloynd-microsoft]

Hi @byronxu99 Can you provide details about this SQL agent and Spider environment, as well as their benefits? The test dir is generally reserved for test scripts to be run by pytest or by users. New agents generally belong under agentchat/contrib.

Hi @rickyloynd-microsoft I moved the agent into the notebook directory as an example, hopefully that's a better place.

Yes, it's simpler. Is this PR just adding a notebook example of how to use SpiderEnv through function calling and SQL? The loop at the bottom of the nb would be convenient if it were inside a .py file, but in the nb it would be better to replace the for loop with a few example turns to illustrate and document the responses that are expected. The other notebooks are organized this way.

Where is spider_env defined? Did that file get dropped from the PR?

At the top of the PR, there should be some explanation of the benefits. ("Why are these changes needed?")

[afourney]

Is this the Spider nl2sql benchmark? That would probably fit better under autogenbench

[byronxu99]

@rickyloynd-microsoft Hi, we have updated the notebook and PR description. spider_env is a Python pip package, and the notebook provides the command for installing it.

@afourney This PR will just be the demo notebook. Any integration with autogenbench would be for future work.

The PR should now be ready to be merged.

[sonichi]

Could you add a link in https://microsoft.github.io/autogen/docs/Examples ? That will help with the discoverability.

[BeibinLi]

I am okay with this notebook. However, the agent is designed for "Spider" benchmark, which is not used generally for actual product. For instance, when users want to use a SQL agent, they would not expect to give "reward" or "score" for the answer, as they may not even know the answer.

So, I would suggest to include the name "spider" into the notebook file name.

[byronxu99]

@sonichi @BeibinLi The requested changes have been made.