Make a SlowFixtureTestCase

[erikrose]

trafficstars

Some models depend on post-save and post-delete hooks to update elasticsearch indices. Consequently, the very act of fixture loading is what populates ES. ES, obviously(?), can't be captured by a transaction, so any test which deletes a model instance could screw up the ES contents for future ones. This makes it necessary to reload the fixtures (or reindex all model objects, or something) before each test. (Corollary: if you're going to use fixtures for ES tests, keep them really small.)

I'm sure there's a crazier, more efficient approach that would minimize duplicated computation—for instance, keep track of which objects tests delete or change, and reindex only those later—but for now, a quick fix might be to provide a SlowFixtureTestCase which gives us all the correctness improvements (locale-resetting and mailbox-resetting) but does not do the FastFixtures stuff. The default TestCase class might as well continue to subclass FastFixtures, since it's working fine for the majority of our cases, but there should be a facility for using our correctness improvements without the per-class fixture loading.

[kumar303]

This sounds good but we also need to solve issue #10 -- a way to somehow opt out of the fixture bundling process altogether while not disturbing other bundles. As SlowFixtureTestCase is described here, I don't see how it will solve that. I did a lot of debugging to figure this out :) and it's hard to explain but I can also jump on skype to go through the ESTestCase scenario.

[erikrose]

SlowFixtureTestCase would not do any bundling, as not doing per-class fixture loading implies not doing bundling. Is there more I'm missing? Happy to Skype at some point if so.

[kumar303]

It's here in the code: https://github.com/jbalogh/django-nose/blob/master/django_nose/fixture_bundling.py#L85

Imagine if fixture_bundle[0] is a SlowFixtureTestCase class. It will not have its fixtures loaded (which is expected). But now all other classes in the group will be set to False so their fixtures won't load either.

[erikrose]

Right. I'm saying "Let's do something to SlowFixtureTestCase (whether adding a dummy fixture or a dedicated flag) so the bundler doesn't bundle them." :-)

[erikrose]

Actually, scratch that. That's what I get for commenting on things 2.5 minutes after rolling out of bed. :-)

First of all, good point in https://github.com/jbalogh/test-utils/issues/9#issuecomment-1996043; that'll save us serious grief.

Second, there are 3 frequencies of fixture loading possible:

1. Per bundle
2. Per class
3. Per test

Number 1 is obviously useful. 3 is necessary for tests like ESTestCase, where tests may trash extra-transactional state. Is 2 ever useful? Maybe in a case where non-transactable setup is done per class (like the current ESTestCase that doesn't support mutation done within a test)?