ormar icon indicating copy to clipboard operation
ormar copied to clipboard

Skip data validation when fetching data from the DB

Open Abdeldjalil-H opened this issue 2 years ago • 6 comments

Problem Desctription

I have noticed that it takes a long time to return the result of simple query. At first glance, I thought it is because of some unnecessary joins. But it turns out that the queries are too fast (less than 10ms), while creating and validating pydantic models take a lot of time (~600ms).

Proposed solution

When I check pydantic docs I find this:

pydantic also provides the construct() method which allows models to be created without validation this can be useful when data has already been validated or comes from a trusted source and you want to create a model as efficiently as possible (construct() is generally around 30x faster than creating a model with full validation).

I changed the method ModelRow.from_row to use construct method:

- instance = cast("Model", cls(**item))
+ instance = cast("Model", cls.construct(**item))

this significantly reduces response time. I think it is safe to skip data validation since it has been validated at creation time. If that sounds reasonable I can create a PR to fix this.

Abdeldjalil-H avatar Sep 29 '22 16:09 Abdeldjalil-H

That's not so simple, unfortunately. A lot of things happen in __init__ of ormar models, among them registration and population of relations. Probably there is a chance to skip fields validation on loads (it already happens for pk_only models) but you cannot just switch to construct.

collerek avatar Oct 02 '22 16:10 collerek

@collerek The method construct calls _construct_relations. So this will create the relations. There is a bug in _construct_relations but this can be fixed.

Abdeldjalil-H avatar Oct 03 '22 16:10 Abdeldjalil-H

Ah ok, I forgot I already did inherit construct method 😅 Yeah we can try that (switching to construct in from_row I mean).

If you found a bug please report it in a separate issue to handle it first.

Thanks!

collerek avatar Oct 04 '22 09:10 collerek

Hey @collerek @Abdeldjalil-H any updates on this ? Just curious to knw. Thanks.

blazing-gig avatar Oct 10 '22 08:10 blazing-gig

@blazing-gig I am waiting for pr #870 to be merged. After that I'll try to work on it. There are about 22 tests that are failing.

Abdeldjalil-H avatar Oct 10 '22 08:10 Abdeldjalil-H

Has there been any update on this? I'd love to be able to skip validation on load.

vaibhavsagar avatar May 28 '24 01:05 vaibhavsagar