chore(django): improve snapshot tests

[Kyle-Verhoog]

The non-asgi snapshots previously used the pytest-django app to generate snapshots. This somewhat worked but had the drawbacks of not knowing what pytest-django did during startup and teardown (like doing database clean up and tear down) which could generate spans and cause the snapshots to fail.

The solution is to move the non-asgi snapshots to use real django applications spawned in subprocesses (like the asgi snapshots). This should help make the tests less flaky as well as give us the flexibility in the future to more easily add new django apps.

https://pymotw.com/2/subprocess/#process-groups-sessions was a handy resource.

[codecov-commenter]

Codecov Report

Merging #3705 (c96b71a) into 1.x (85c3c96) will decrease coverage by 0.03%. The diff coverage is 82.40%.

@@            Coverage Diff             @@
##              1.x    #3705      +/-   ##
==========================================
- Coverage   78.08%   78.05%   -0.04%     
==========================================
  Files         646      647       +1     
  Lines       49995    50028      +33     
==========================================
+ Hits        39039    39049      +10     
- Misses      10956    10979      +23     

Impacted Files	Coverage Δ
tests/contrib/django/django1_app/urls.py	100.00% <ø> (ø)
tests/contrib/django/django_app/settings.py	100.00% <ø> (ø)
tests/contrib/django/django_app/urls.py	100.00% <ø> (+7.40%)	:arrow_up:
tests/contrib/django/manage.py	0.00% <0.00%> (ø)
tests/webclient.py	94.28% <ø> (ø)
tests/contrib/django/views.py	85.21% <14.28%> (-11.82%)	:arrow_down:
tests/conftest.py	95.32% <88.88%> (-0.42%)	:arrow_down:
tests/contrib/django/test_django_snapshots.py	96.49% <95.74%> (-3.51%)	:arrow_down:
tests/contrib/django/django1_app/settings.py	100.00% <100.00%> (ø)
tests/contrib/django/django_app/extra_urls.py	87.50% <0.00%> (-12.50%)	:arrow_down:
... and 1 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[Kyle-Verhoog]

I've re-run the Django job a few times now and all 🟢 so 🤞 this addresses the flakiness!

[majorgreys]

@Kyle-Verhoog If you re-enable the middleware spans now, do you find the flakiness returns?

[majorgreys]

Should we expect this to change much the total duration?

[Kyle-Verhoog]

@Kyle-Verhoog If you re-enable the middleware spans now, do you find the flakiness returns?

@majorgreys there's no difference, the middleware isn't responsible for the flakiness. The flakiness was due to pytest-django doing setup/teardown with databases as well as a race condition that I found here: https://github.com/DataDog/dd-trace-py/pull/3705/files#diff-003ab1c6f93cd945d1ba16bbbcecd5d41471bc72a2b99305cd8c117471204c95R112

I cut out the middleware just to make the snapshots a bit more readable.

[Kyle-Verhoog]

Looks like there is some flakiness still: https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/17170/workflows/bc9ec7ff-784c-4eae-9912-b040eed74883/jobs/1171765

Looking into this one

[Kyle-Verhoog]

😩 still some flakiness present

==================================== ERRORS ====================================
________________ ERROR at teardown of test_urlpatterns_include _________________
At request <Request GET /test/session/snapshot >:
   At snapshot (token='tests.contrib.django.test_django_snapshots.test_urlpatterns_include_21x'):
    - Directory: /snapshots
    - CI mode: 1
    - Trace File: /snapshots/tests.contrib.django.test_django_snapshots.test_urlpatterns_include_21x.json
    - Stats File: /snapshots/tests.contrib.django.test_django_snapshots.test_urlpatterns_include_21x_tracestats.json
    At compare of 1 expected trace(s) to 0 received trace(s):
Did not receive expected traces: 'django.request'

[mergify[bot]]

@Kyle-Verhoog this pull request is now in conflict 😩

[mergify[bot]]

@Kyle-Verhoog this pull request is now in conflict 😩