chainlit
chainlit copied to clipboard
[Draft] Proposal for SQLAlchemy Data Layer with ORM
This is a proposal for a different way to implement SQLAlchemy Data Layer using ORM instead of raw SQL statements, taking inspiration from https://github.com/Chainlit/chainlit/pull/832 Hopefully it will simplify support for other SQL dialects besides Postgres and SQLite.
This is a rather major change:
- [x] Define tables and indices using SQLAlchemy objects
- [x] Use SQLAlchemy functions to create statements instead of raw SQL queries
Other changes:
- [x] in
get_thread
andlist_threads
. only get specific thread(s) for better performance and reduced bandwidth compared toget_all_user_threads
(note: reduced bandwidth is important for providers such as Supabase) - [x] in
list_threads
, filter would use thread name as well as steps input/output texts, this needs careful review. - [x] in
create_element
, storage_provider is made optional so that elements that don't need content to be stored will still be added to database, example of these elements are external links that can be read and displayed at runtime.
The diff between current SQLAlchemy and this one can be found here: https://gist.github.com/qtangs/71370e6ce5feec78d9586c808804e81c/revisions?diff=split&w=
Some tests have been added based on https://github.com/Chainlit/chainlit/blob/main/backend/tests/data/test_sql_alchemy.py but more might be needed.