Consider filtering on key_fields

[henribru]

Currently all objects that match filters are fetched from the database. If there are a lot of these and few matching objects in new_models, this isn't very efficient. It seems like it should be possible to only fetch objects that match an object in new_models by filtering on key_fields. Basically you would build up a large condition akin to (col1 = new_model1.col1 AND col2 = new_model2.col2) OR (col1 = new_model2.col1 AND new_model2.col2) OR ... using Q objects and use that as a filter.

Of course if every object in new_models already exists in the database anyway, this is going to be less efficient, so there's a trade-off. I haven't benchmarked the idea to see in which situations it would be beneficial, but it might be worth looking into.

[ses4j]

I'm not sure I follow everything you're describing. However, it only fetches the primary key of each object from the db, not the whole object. Also, you need to know what is in the db in your "key space" that is NOT in new_models, so you know what to delete. Does that alter your thinking?

[ses4j]

Closing as inactive.