upgrade-util icon indicating copy to clipboard operation
upgrade-util copied to clipboard

[IMP] pg,orm,models: query ids server side

Open cawo-odoo opened this issue 1 month ago • 4 comments

[ADD] pg.query_ids: query large numbers of ids memory-safely

This is mainly the code that has been recently added to orm.recompute_fields, here we're making it re-usasble.

[IMP] orm.recompute_fields: use new pg.query_ids

This code in recompute_fields has been made re-usable in a new util pg.query_ids. Use that.

[FIX] models.remove_model: MemoryError

Traceback (most recent call last):
[...]
  File "/tmp/tmpipxrg2eq/migrations/util/models.py", line 563, in merge_model
    remove_model(cr, source, drop_table=drop_table, ignore_m2m=ignore_m2m)
  File "/tmp/tmpipxrg2eq/migrations/util/models.py", line 138, in remove_model
    it = chunks([id for (id,) in cr.fetchall()], chunk_size, fmt=tuple)
MemoryError

Some IR tables can be large. Avoid cr.fetchall() when getting ids by use of pg.query_ids()

cawo-odoo avatar Nov 24 '25 14:11 cawo-odoo

Pull request status dashboard

robodoo avatar Nov 24 '25 14:11 robodoo

upgradeci retry with always only crm hr

KangOl avatar Nov 24 '25 15:11 KangOl

so, this avoids the memoryerror in upg-3307099 just fine

cawo-odoo avatar Nov 25 '25 14:11 cawo-odoo

I pushed a version inspired by your suggestions, but I didn't use them verbatim. Please resolve conversations when you feel they have been addressed.

2 explanations up-front:

  • It does not work to raise a real exception from __iter__ when the cursor is closed, because util.chunks works that way that it needs to catch StopIteration twice before it really stops iterating.
  • afaik, returning a value from __del__ is useless, it is ignored always

cawo-odoo avatar Nov 26 '25 14:11 cawo-odoo