upgrade-util
upgrade-util copied to clipboard
[IMP] pg,orm,models: query ids server side
[ADD] pg.query_ids: query large numbers of ids memory-safely
This is mainly the code that has been recently added to orm.recompute_fields, here we're making it re-usasble.
[IMP] orm.recompute_fields: use new pg.query_ids
This code in recompute_fields has been made re-usable in a new util pg.query_ids. Use that.
[FIX] models.remove_model: MemoryError
Traceback (most recent call last):
[...]
File "/tmp/tmpipxrg2eq/migrations/util/models.py", line 563, in merge_model
remove_model(cr, source, drop_table=drop_table, ignore_m2m=ignore_m2m)
File "/tmp/tmpipxrg2eq/migrations/util/models.py", line 138, in remove_model
it = chunks([id for (id,) in cr.fetchall()], chunk_size, fmt=tuple)
MemoryError
Some IR tables can be large. Avoid cr.fetchall() when getting ids by use of pg.query_ids()
upgradeci retry with always only crm hr
so, this avoids the memoryerror in upg-3307099 just fine
I pushed a version inspired by your suggestions, but I didn't use them verbatim. Please resolve conversations when you feel they have been addressed.
2 explanations up-front:
- It does not work to raise a real exception from
__iter__when the cursor is closed, becauseutil.chunksworks that way that it needs to catchStopIterationtwice before it really stops iterating. - afaik, returning a value from
__del__is useless, it is ignored always
