Misago icon indicating copy to clipboard operation
Misago copied to clipboard

Replace chunk_queryset() util with server-side cursor

Open metalerk opened this issue 6 years ago • 2 comments

Replace chunk_queryset() util with server-side cursor #1230

Motivation

Currently Misago uses chunk_queryset utility that iterates on queryset in chunks of specified length, but this task can be easily replaced with queryset.iterator() which on PostgreSQL should use chunking in database.

Solution

Change function util chunk_queryset to iterator() method.

metalerk avatar Jul 31 '19 01:07 metalerk

chunk_size parameter is set to 20 in most cases because in chunk_queryset the default was 20 and iterator default is 2000

metalerk avatar Jul 31 '19 02:07 metalerk

I see that some tests exploded after this change :x

Is there also a way to confirm that iterator() is walking the table directly, and not creating temporary result table in PostgreSQL that is copy of main table?

rafalp avatar Aug 02 '19 22:08 rafalp