[Proposal] Scheduled daily github workflow to run tests using Django's main branch
Was listening to DjangoChat and how we don't get enough testers for pre-releases. It was also mentioned that folks may not know how to do this
I think djangoproject.com should lead by example. We should have our tests run against Django main everyday with warnings enabled. Other folks can then look at this workflow as an example for their own project. We are also contributing to testing pre-released Django 👍
I like the idea.
I think djangoproject.com should lead by example
Yes, but I don't think that best practice applies to this project. One important point is that this repo is a project, not a library. It will not be used with multiple Django versions at any given time, so I think it's perfectly okay to have a single version in tests.
Also, it's not clear how long we will be able to keep this project up to date with the latest version of Django, due to Python compatibility issues with Trac. I'll recommend finalizing
- https://github.com/django/djangoproject.com/pull/2234
- https://github.com/django/djangoproject.com/issues/2265
first so that we can have a healthier discussion.
Django projects, not just libraries, should test pre-releases ideally on a regular basis 🙂 I think a test matrix doesn't fit this use-case so well, as there is no need to have this run on every PR. It's more about having our tests run periodically against Django main (could even be once a week, once a month etc) to test that Django itself hasn't accidentally broken something for our project
I don't disagree with the intention, but I still don't think it's needed or beneficial. We can always create a new branch with a new Django version (or main) and can run tests on it.
I agree that it's not needed. More of a nice-to-have than a true need 👍
In case we are discussing the benefit of doing anything at all, then the benefit is, if something got merged into Django that broke some existing functionality accidentally that we relied on, we would catch this early and raise a ticket. They would then fix it before that breaking change is even included in a release. Otherwise, we would need to wait a month for a patch release of Django before we would upgrade versions (if we were only checking once a final release is out). That's the benefit of anybody (whether it's a library or a normal Django project) testing against Django main on a regular basis. Having a scheduled job would mean this happens automatically. In theory, someone could do this manually every now and again
However, you may be trying to discuss an implementation detail rather than the benefit of doing something here? I'm open to other ways to approach this :)
Things to consider:
- that something is broken on Django main but works on the current version of Django we are using should not block a PR, hence I wouldn't have Django main tests ran on PRs
- how regular doesn't matter too much, the more regular makes it easier to find the breaking commit of Django
- I prefer having something automatically scheduled than trusting some folks would do this locally manually every now and again
@sarahboyce I really do understand where you are coming from, and I'd just approve this if we were talking about a project that is applying updates often and quickly. Given the project's development experience, I don't see a case where we would have to struggle with an issue that could have been caught earlier, thanks to being tested with main.
My vote will be negative, so we won't be adding unnecessary complexity. If the other people supports the idea and see no issues with the additional maintenance load, I'll comply with the majority.
@sarahboyce Thanks for putting this forward. I think using the djangoproject.com repo to demonstrate testing against pre-releases is a great idea. In his lightning talk at DjangoCon US 2025, @adamzap suggested this repo should be an example for others; I trust this will inspire more developers to test pre-releases and provide feedback to Django. Perhaps it's even worth a blog post on djangoproject.com? Regardless, I plan to use this workflow as a reference for my own projects. Thanks!
I absolutely agree with Tobias and Sarah. I think this proposal has clear benefits, including:
- As Tobias and Sarah noted, using the djangoproject.com repository to demonstrate testing against Django pre-releases.
- As Sarah said, providing the Django framework with much-needed early testing during the pre-release phase.
- Nice side effect but not necessarily main driver: enabling earlier detection of potential compatibility issues between djangoproject.com and Django releases.
Thank you @sarahboyce for pushing this initiative!
I like this idea!
+1 for some canary testing here!
+1
I only found this issue today and it really made me smile because yesterday I was listening to the same Django Chat episode with Natalia and had exactly the same idea. I even shared a quick thought about it on Mastodon before noticing this issue: https://fosstodon.org/@paulox/115723883286082489
So +1 from me as well, and kudos to Sara for being much faster than me at actually opening the issue, clearly that episode was very inspiring. 😂