Multiple calls to annotate returns only latest annotation when calling select_subclasses.

[digismack]

Problem

When making chained calls to annotate() on a single queryset, only the last annotation applied is returned after calling select_subclasses. If you don't call select_subclasses, you do receive all annotations.

Environment

• Django Model Utils version: 3.1.1
• Django version: 2.0
• Python version: 3.5.2
• Other libraries used, if any: n/a

Code examples

In the sample below, the only annotation that is available in the evaluated queryset is correct_count.

import django
django.setup()  # noqa

from django.db.models import Case, When, ExpressionWrapper, FloatField, Sum, Q

from sampleapp import models

correct_count_calc = ExpressionWrapper(
    Sum(
        Case(
            When((Q(answer__correct=True) & Q(answer__quizresult__user__is_superuser=False)), then=1),
            default=0
        )
    ), FloatField()
)

answered_count_calc = ExpressionWrapper(
    Sum(
        Case(
            When(answer__quizresult__user__is_superuser=False, then=1.0),
            default=0
        )
    ), FloatField()
)

q = models.BaseQuestion.objects.annotate(answered_count=answered_count_calc) \
                               .annotate(correct_count=correct_count_calc).select_subclasses().first()
print(q.__dict__)

It seems the problem stems from the snippet below, which sets (rather than appending to) the qset._annotated value for each subsequent call to .annotate().

https://github.com/jazzband/django-model-utils/blob/1eff6d0d8f5e880db63493b59126a448a30a74e1/model_utils/managers.py#L116

I have a simple fix for this issue here:

https://github.com/digismack/django-model-utils/commit/6a72cebfc02fd19e68c15343fa9647256ad1e0f9

I was unable to get the test suite to run. They all fail with AppRegistryNotReady: Apps aren't loaded yet. If someone would be willing to help me out a bit, I'd love to submit a pull request with tests and such.

[digismack]

@tony Maybe you can take a look at this and let me know what you think?

[kaojistream]

I also encountered this problem.How did you solve it in the end? @digismack

[digismack]

@kaojistream I just ended up creating a fork of the repo with my patch applied and I use it in my project's requirements.txt file instead. Every so often I pull in the latest upstream changes. I'd love to get a patch and tests accepted, but I still can't get the tests to run. 🤷

[areyal]

Hi, just to say thanks, and to indicate that this problem didn't exist for my app with django 3.0. Upgrading to django 3.1 showed this up.

[areyal]

I'm facing some problems with duplicated annotations in your solution. To solve it, i've used this code instead yours in managers.InheritanceQuerySetManager.annotate:

annlist = [a.default_alias for a in args] + list(kwargs.keys())
if not hasattr(qset, '_annotated'):
    qset._annotated = annlist
else:
    qset._annotated += [a for a in annlist if not a in qset._annotated]

This avoids duplicated names for anntations, but the problem with unfound annotations still persists for my app, so I guess there is some deeper cause for Django 3.1.3 + django-model-utils 4.0.0

[kltchnko]

@areyal @digismack Thank you for finding out this issue and possible solutions! I slightly changed the code and it worked (Django==3.1.7, django-model-utils==4.1.1)! You don't need to fork this repo, just subclass from django.db.models.QuerySet and add InheritanceQuerySetMixin.

class FixedInheritanceQuerySet(InheritanceQuerySetMixin, QuerySet):
    def annotate(self, *args, **kwargs):
        qset = super(InheritanceQuerySetMixin, self).annotate(*args, **kwargs)
        annlist = [a.default_alias for a in args] + list(kwargs.keys())
        if not hasattr(qset, '_annotated'):
            qset._annotated = annlist
        else:
            qset._annotated.extend(a for a in annlist if a not in qset._annotated)
        return qset