ormar
ormar copied to clipboard
Skip data validation when fetching data from the DB
Problem Desctription
I have noticed that it takes a long time to return the result of simple query. At first glance, I thought it is because of some unnecessary joins. But it turns out that the queries are too fast (less than 10ms), while creating and validating pydantic models take a lot of time (~600ms).
Proposed solution
When I check pydantic docs I find this:
pydantic also provides the construct() method which allows models to be created without validation this can be useful when data has already been validated or comes from a trusted source and you want to create a model as efficiently as possible (construct() is generally around 30x faster than creating a model with full validation).
I changed the method ModelRow.from_row
to use construct
method:
- instance = cast("Model", cls(**item))
+ instance = cast("Model", cls.construct(**item))
this significantly reduces response time. I think it is safe to skip data validation since it has been validated at creation time. If that sounds reasonable I can create a PR to fix this.
That's not so simple, unfortunately. A lot of things happen in __init__
of ormar models, among them registration and population of relations. Probably there is a chance to skip fields validation on loads (it already happens for pk_only models) but you cannot just switch to construct
.
@collerek
The method construct
calls _construct_relations
. So this will create the relations. There is a bug in _construct_relations
but this can be fixed.
Ah ok, I forgot I already did inherit construct
method 😅
Yeah we can try that (switching to construct in from_row I mean).
If you found a bug please report it in a separate issue to handle it first.
Thanks!
Hey @collerek @Abdeldjalil-H any updates on this ? Just curious to knw. Thanks.
@blazing-gig I am waiting for pr #870 to be merged. After that I'll try to work on it. There are about 22 tests that are failing.
Has there been any update on this? I'd love to be able to skip validation on load.