IntegrityError for polymorphic_ctype_id while running unit tests

[Eboreg]

My setup:

class License(PolymorphicModel)
class ClassLicense(License)
class SchoolLicense(License)

When running manage.py test, I get an IntegrityError during teardown phase of a test that has created one ClassLicense and one SchoolLicense instance:

django.db.utils.IntegrityError: The row in table 'licenses_license' with primary key '2' has an invalid foreign key: licenses_license.polymorphic_ctype_id contains a value '109' that does not have a corresponding value in django_content_type.id.

License with pk=2 is a SchoolLicense. The ClassLicense instance causes no error. 109 is the actual ID of ClassLicense's ContentType in my development SQLite DB. In the temporary test DB it's something else, hence the error.

I did a little digging and found out this was caused by ContentTypeManager's cache being populated with some wrong values. This, in turn, was because I have a form containing this field:

class SchoolClassAdminAddForm(forms.ModelForm):
    school_license = forms.ModelChoiceField(
        queryset=License.objects.instance_of(SchoolLicense, EduInstLicense),
        ...

That is, License.objects.instance_of() gets evaluated during "upstart", and SchoolLicense content type (from my dev DB) gets added to ContentTypeManager's cache.

I can prevent this by running ContentType.objects.clear_cache() at a suitable moment during the test phase, so the cache gets populated with the correct values. But perhaps there is a way to fix this altogether? Or maybe this info should be included in the docs, perhaps in a special 'Testing' document.

[AdamDonna]

@Eboreg Did you have a failing unit test I could use to replicate this? From my reading of this report it sounds like the implementation details are leaking into how it's tested which isn't fun, but i'd like to see the test to confirm this and consider possible solutions.

[Eboreg]

@AdamDonna I'm not comfortable with sharing much of the actual code since it's not open source and belongs to my employer. But it should be possible to replicate by setting up a barebones Django app with the conditions in my post. Maybe I'll have a go at that. I'll let you know in that case.

[Eboreg]

@AdamDonna Of course, if you're only asking for the actual test, it looks like this:

class LicenseViewTests(TestCase):
    [...]
    def test_do_not_send_expiration_reminder_has_newer_license(self):
        _today = today()
        school_class = self.school_classes[0]
        # This one worked:
        lic = school_class.licenses.create(
            start_date=_today,
            end_date=_today + timezone.timedelta(days=7),
        )
        reminder = LicenseExpirationReminder.objects.create(license=lic, date=today())
        # This one caused error during teardown:
        school_class.school.licenses.create(
            start_date=_today,
            end_date=_today + timezone.timedelta(days=365),
        )
        call_command("sendlicensereminders", stdout=StringIO(), stderr=StringIO())
        self.assertEqual(len(mail.outbox), 1)
        recipients = [email for sublist in mail.outbox for email in sublist.to]
        managers = [m[1] for m in settings.MANAGERS]
        for manager in managers:
            self.assertIn(manager, recipients)
        reminder.refresh_from_db()
        self.assertFalse(reminder.sent)

school_class.licenses = ClassLicense objects, school_class.school.licenses = SchoolLicense objects.

After I put ContentType.objects.clear_cache() in LicenseViewTests.setUp(), the problem was solved.

[AdamDonna]

@Eboreg Yep, of course, I wasn't asking you to share specifics. I was just after enough to replicate the issue and get a PR started. I think the only things i need answered are

1. What does the LicenseExpirationReminder look like?
2. Is school class list created in setUp or setUpClass ect... I think that lifecycle is important here
3. Does Teardown to anything special?

[Eboreg]

@AdamDonna

1. LicenseExpirationReminder looks like this:

class LicenseExpirationReminder(models.Model):
    license = models.ForeignKey(
        "License", on_delete=models.CASCADE, related_name="expiration_reminders", verbose_name=_("license"))
    date = models.DateField(_("date"), help_text=_("Send reminder on this date."))
    sent = models.BooleanField(_("sent"), help_text=_("Whether reminder has been sent."), default=False)

No overriding of __init__(), save() or other methods.

2. self.school_classes in the test above are created in setUp.

3. The teardown process is only Django's own. I haven't done any alterations to it.