Rewrite AutoGen Studio Database Layer to Use SQLModel ORM

[victordibia]

Why are these changes needed?

This PR rewrites the database module for autogenstudio using use SQLModel (Pydantic + SQLAlchemy). This provides

• [x] Entity linking (skills, models, agents and workflows are linked via association tables). This addresses multiple existing issues where changes to skills, models or agents do not propagate to workflows.
• [x] Supports for multiple database backend dialects supported in SQLAlchemy (SQLite, PostgreSQL, MySQL, Oracle, Microsoft SQL Server). The backend database can be specified a --database-uri argument when running the application. For example, autogenstudio ui --database-uri sqlite:///database.sqlite for SQLite and autogenstudio ui --database-uri postgresql://user:password@localhost/dbname for PostgreSQL.
• [x] Adds support for LocalCommandLineCodeExecutor and DockerCommandLineCodeExecutor introduced by @jackgerrits and @ekzhu

Note: This PR introduces significant changes to the database schema, incompatible with v< 0.0.56.

TODO

• [ ] Add initial tests for the database manager
• [x] Add some database migration logic (e.g., detect incompatibilities and take some action)

Testing

Install autogenstudio from autogenstudio-orm branch. Run autogenstudio ui

Related issue number

#1694 #2128

Checks

• [x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
• [ ] I've made sure all auto checks have passed.

[Knucklessg1]

Wow, this is fantastic! Happy to see support for more DBs!

[ekzhu]

@ma-armenta I have invited you to join as a collaborator to review this issue.

[ekzhu]

When I start AutoGen studio built from this branch, it shows multiple "Connection error 404 Not Found" in a banner error message.

[ShaneYuTH]

@victordibia The field "admin_name" is declared twice with the exact same specifications and default value from AgentConfig under datamodel.py. version: autogenstudio 0.0.56rc1

[victordibia]

@victordibia The field "admin_name" is declared twice with the exact same specifications and default value from AgentConfig under datamodel.py. version: autogenstudio 0.0.56rc1

Thanks. Fixed.

[ShaneYuTH]

@victordibia Unable to delete sessions, got "The Session is linked to another entity and cannot be deleted". version: 0.0.56rc3

[ShaneYuTH]

@victordibia Unable to delete sessions, got "The Session is linked to another entity and cannot be deleted". version: 0.0.56rc3

FYI: It violates foreign key constraint "message_session_id_fkey" on table "message"

[ma-armenta]

@victordibia I am building from source to test things out. Using postgresql and got ModuleNotFoundError: No module named 'psycopg2'. Lib is installed, however I have been using psycopg2-binary instead to avoid any compiler or external library requirements, if any, just some thoughts.

[gitguardian[bot]]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
10404662	Triggered	Generic CLI Secret	2448baa5bb8a1de3cb78303219f23cbfdb46dfc3	.github/workflows/dotnet-release.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}