fastapi
fastapi copied to clipboard
tiangolo
[FEATURE]Consider add support tortoise-orm?
Which support do you mean? FastAPI doesn't depend on any orm, you can use what you want.
If anyone has used (or tried to use) tortoise-orm in combination with FastAPI, I'd be interested to hear about their experience.
I think it would make sense to add a reference to a blog article or similar if someone wrote one about using the two together.
I found that peewee has been introduced into FastAPI's documentation.
https://fastapi.tiangolo.com/tutorial/sql-databases-peewee/
There is a PR in progress that should make this easier: https://github.com/tortoise/tortoise-orm/pull/260
And here is a FastAPI + Tortoise ORM implementation: https://frankie567.github.io/fastapi-users/configuration/databases/tortoise/
I would really like to get it all working as I like the design of FastAPI, but isn't using it in any of my current work/projects. So any suggestions as to lowering the barrier of entry is welcome :-)
Also you can see this: https://github.com/prostomarkeloff/fastapi-tortoise
@jet10000 about the original question, yes, you can use Tortoise directly, it should work.
About Pydantic support directly in Tortoise, wow! That would be huge! :rocket:
Given that, we should definitely have a section in the docs dedicated to Tortoise after this is merged: https://github.com/tortoise/tortoise-orm/pull/276 :memo:
Having automatic Pydantic from ORM (or vice versa) has been a frequently-requested feature (although it's out of scope for FastAPI itself).
@timothycrosley sorry to spam you again, this is another example of these ideas, it might be closer than I thought!
We have finally released the first version of Tortoise ORM with pydantic serialisation support, and it is documented here: https://tortoise-orm.readthedocs.io/en/latest/contrib/pydantic.html
Could I ask that you please have a look. We are only doing serialisation from Tortoise at this stage, but plan to add Create/Update/Merge operations soonish. (I rather wanted it to work well before we start adding more features).
We have an open issue where we are discussing next steps for this: https://github.com/tortoise/tortoise-orm/issues/315 where you are welcome to participate in.
Also, at this stage we only optimised some low-hanging fruit, and an entire sample app that we can benchmark against has not been done. I would like to be closer to feature complete before I do that — as in do an entire LCRUD sample, and benchmark it using Locust or some other scriptable benchmarking system.
Just an update, Even though what we did works fine for pydantic, fastapi handles things in specific ways:
- We need a better way of handling
List[]of item. (have a plan) - For create (and possibly update) we need to exclude read-only fields. https://fastapi.tiangolo.com/tutorial/extra-models/ proposes using inheritance to do that, but now we have to return multiple models, e.g. read-model & write-model. I can't think of an easy way to do this off the top of my head :thinking:
- Model names must be globally unique (we didn't take that into account)
I'm fixing these in my quest to build a clean example, that can handle a bit more than the basic stuff.
@grigi that's awesome! :rocket:
[...] using inheritance [...]
Have in mind that that is only to reduce duplication. But the JSON Schemas are generated as if they were completely independent.
So, if generated models are multiple, for read, for write, etc. That's perfectly fine. The only reason to use inheritance is to avoid developer-typed code repetition. But if the model is generated that's perfectly fine.
Model names must be globally unique
I haven't been able to check how it would work with your new features, but we could think of ways of generating names for models generated on the fly or something similar, and this is probably not going to benefit only Tortoise-ORM, but others as well, e.g. someone could build a package that generates Pydantic models from SQLAlchemy models, and those would be generated as well.
So, the FastAPI side is not totally rigid :smile: If it makes sense, we can think of ways to update it.
There's a subtlety that I think it's easy to miss (but I think you caught it).
Pydantic itself doesn't do any async stuff. So, it can serialize already received data, but not fetch it using await automatically.
But if I understand correctly, the model would be generated from Tortoise to document and serialize the API and the path operation functions would serialize it with the generated model before returning, right?
So, something like:
from fastapi import FastAPI
from tortoise import fields
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise.models import Model
class Tournament(Model):
"""
This references a Tournament
"""
id = fields.IntField(pk=True)
name = fields.CharField(max_length=100)
#: The date-time the Tournament record was created at
created_at = fields.DatetimeField(auto_now_add=True)
Tournament_Pydantic = pydantic_model_creator(Tournament)
app = FastAPI()
@app.get("/tournaments/{id}", response_model=Tournament_Pydantic)
async def read_tournament(id: int):
tournament = await Tournament.get(id=id)
return Tournament_Pydantic.from_tortoise_orm(tournament)
Thank you :moon:
Have in mind that that is only to reduce duplication. But the JSON Schemas are generated as if they were completely independent.
Ok, that's fine then. I did much the same.
Model names must be globally unique
I haven't been able to check how it would work with your new features, but we could think of ways of generating names for models generated on the fly or something similar, and this is probably not going to benefit only Tortoise-ORM, but others as well, e.g. someone could build a package that generates Pydantic models from SQLAlchemy models, and those would be generated as well.
That could work. The only other solution I could think of is to generate the model reference name (that needs to be unique) to be a hash. Which will work, just create somewhat messy schemas. I am doing this right now for self-referential models, but that could be solved better.
Self referential models are done by creating sub models N-levels deep, each with metadata re db query to be created. This works well to limit depth, but the depth is somewhat static now.
Using a hashed name for the exact config of the model would get rid of conflicts. e.g. Tournament_af3e7 as an internal de-duplication so we can re-use schemas more effectively?
There's a subtlety that I think it's easy to miss (but I think you caught it).
Pydantic itself doesn't do any async stuff. So, it can serialize already received data, but not fetch it using
awaitautomatically.
Yes, I ensure that all the I/O awaiting is done before I call Pydantic. I treat it much like one should treat templating. For predictable behaviour, no I/O should be done in the presentation layer.
I really would have gotten further with this, but got sick (and was basically sleeping for 2½ weeks) then the whole country gone into lockdown (and I have to help entertain the kids) So time has been limited the last, er. 5 weeks? And at least the next 3. And then existing PR's and bugs take priority :shrug:
Just an update on Tortoise's Pydantic integration.
I got some time to work on it some more, and I now generate globally unique definition indexes that is in <FQDN>.<6-char hash> format. (hash is mime32 alphabet, lowercase, so 30 bits effective)
The hash is dependant on entire model and depth-stack, so the same object can appear multiple times, but they have different contexts.
Leaf-node schemas (e.g. default model, but excluding all relations as children) is a detected special case, and given the special hash of leaf, this is to minimise the schema size. I found it reduced some schemas by ⅓ in size.
I also don't have a hash portion if the schema is entierly default, and it is the root.
@tiangolo Is such naming convention good?
Some examples:
-
tests.testmodels.Reporter.leaf -
tests.testmodels.Event.b4oydv -
tests.test_early_init.Tournament
I specify a title of just the base name, e.g. Reporter, Event, etc...
I still am going to add aliases so one can rename some entries. Then I think it will be basically feature complete.
How do we go about of adding documentation to FastAPI? I have some basic Pydantic tutorials here: https://tortoise-orm.readthedocs.io/en/latest/contrib/pydantic.html#contrib-pydantic and a simple FastAPI example here: https://tortoise-orm.readthedocs.io/en/latest/examples/fastapi.html
Exciting news!!! And how to write test code when FastAPI and tortoise-orm cooperate? for example: some testcases need use the Restful API to insert data ,then query results by ORM.
Looking forward to very much, If I can do anything, please let me know.
The testing thing is going to be interesting. Using the proposed system to do a request/response ONLY is going to work. but having an async context available where you can edit/check on state and mix it with the request/response tester... the issue is the test runner creates a new event loop for every test, and the DB connection expects a single event loop. Also, only one event loop per thread is allowed, so you'll get multiple-event-loop errors before you even got to running the tests.
Also, creating a new loop and booting up all async connections for every test is just terribly slow. But we need to get it working first. The solution will likely require changing the test runner...
Just for completeness sake: We have a basic test example here: https://tortoise-orm.readthedocs.io/en/develop/examples/fastapi.html#tests-py
It's not perfect (as in it doesn't "flow" nicely) But it works.
@dmontagu I have been using Fastapi with Tortoise-orm for relational stuff for last few months.
Don't really have time as of now to write blog. So some short notes of my experience:
I would usually declare the pydantic schema for the create requests and inherit it for responses(which contains id etc). Then in the views, I return a dict either using the .values() for filters or .__dict__ for single object.
Lack of timezone support sucks and I had to declare datetime fields without auto_now or auto_now_add, which I extensively used in django. Then I would have to manually modify the table and change the field type from timestamp to timestamptz and in the code, populate those fields using datetime.now(pytz.utc).
Another issue is migrations. Tried to integrate Aerich as it seemed as the only alternative but couldn't downgrade easily(https://github.com/tortoise/aerich/issues/72).
Another problem I faced was using celery with Tortoise-orm. Even with celery 5. I just couldn't make it work as celery doesn't support asyncio and using asyncio or asgiref was raising some pool closed error. As a quick workaround, I just made an internal REST endpoint in the fastapi app which calls the async related stuff and called them from the tasks. There's definitely a better way, but that's for another thread.
@anindyamanna , can you example how can i make relation correctly ? I used ForeignKeyField, but no appear column on swagger and appear error:
{ "detail": [ { "loc": [], "msg": "(1048, "Column 'membro_group_id' cannot be null")", "type": "IntegrityError" } ] }
