Better first-class Django support

[Allan-Nava]

I want to versioning my django project with pants, is possible?

[Eric-Arellano]

Bump that we still need to do this.

[danxmoran]

Some Django things I've hit that don't Just Work in Pants:

• Inference based on INSTALLED_APPS. Django supports the short-hand where you can specify just an app's package name and it knows to look for an apps.py within that package. Pants doesn't know about this convention, so inference only works if you use the long-hand and fully specify the name of your AppConfig subclass.
• Dependency inference from apps.py to models.py (or models/__init__.py), migrations/, fixtures/ (maybe others?)
• Dependency inference for admin.py (or admin/__init__.py) in each of your apps if django.contrib.admin is in the INSTALLED_APPS
• Dependency inference for translation.py (or translation/__init__.py) in each of your apps if modeltranslation is in the INSTALLED_APPS

[kaos]

I've been looking at a Django project for a few weeks now, and given the above I think would make the most sense to do, is open up the Pants dependency inference engine to some configurability (or alternatively have a django backend that adds the relevant rules for the specifics in this case).

For the generic case solving with configuration I was considering to look at more strings in the sources for what is paths/modules to infer deps on. Not sure on syntax etc for that yet though.

[Eric-Arellano]

and given the above I think would make the most sense to do, is open up the Pants dependency inference engine to some configurability

It's very pluggable already. I think everything we need is there from a plugin perspective, we only need to implement it.

or alternatively have a django backend that adds the relevant rules for the specifics in this case).

I suspect this is a good route. Many many Python users don't use Django at all, e.g. data science teams. We don't want to make their experience more complex for something they don't use. Generally, v2 tries to be opt-in rather than opt-out

[asherf]

I double checked stuff in our source code (toolchain) which is a Django monorepo w/ multiple Django apps composed into microservices. We hit all of those issue (except the translation stuff since we don't translate our web app yet).

• INSTALLED_APPS issue, we do end up specifying the class name, for example: "toolchain.jerry.apps.JerryAppConfig", I don't find it particularly hard to do, since personally I prefer explicit rather than implicit.

• usually models are referenced from other places in the code (views) so I never had a case where we had a missing dependency (ModuleNotFound) issue there. as for migrations, we do need to add an explicit dependency on that directory to the service that we use the run the migration (in our solution, we only need to deploy migration to a centralized Django admin service where we run them so the don't need to be a part of each microservice). We do hit an issue w/ Django management commands where we need to add them explicitly to deployed services (since nothing depends on them). This is one of those things I keep forgetting to do, so I deploy a service and try to run a command only to discover it is not there.

• Dependency or automatically including the Django admin stuff in various services/apps actually kind of freaks me out (since it can create a huge security issue). For us, the solution we came up with is to have a cerntelized admin service for all of our micro services and to not deploy the Django admin stuff w/ every service/app.

I think the solution here is to allow the dependency inference infra in pants to be pluggable so people can enable a plugin that deals with Django specific stuff (and presumable other frameworks, web or otherwise have their own things) Having a Django specific dependency inference builtin to pants (but not enabled by default) that will deal with the issues mentioned here makes a ton of sense to me. It will also serve as an example to customize dependency inference for other use cases. FWIW, at least for python, this plugin API should be more like the flake8 plugin API (i.e. passes various AST objects to the plugin) rather than the standard pants plugin API (rules and such) which I find harder to grok, debug and test.

[Eric-Arellano]

Thanks @asherf!! I think what this would ideally look like is a Django backend, and it adds a subsystem that allows you to turn on & off various dependency inference rules. So, you can turn off the admin rule because Toolchain like the implications of it, but others can still use it.

And then maybe we want to optimize other workflows? For example, I wonder if we could make Django testing better (both w/ Pytest and their normal Django test runner)

[danxmoran]

Another Django-ism we've implemented in our repo is a replacement for django-admin compilemessages (docs here). I looked through the Django source and it turns out that command is a thin wrapper around the msgfmt binary from gettext - it was very easy to reproduce as a Pants codegen plugin. I'd be happy to upstream it.

[benjyw]

I do think @asherf's idea of a higher level API specifically for dep inference makes sense.

[github-actions[bot]]

This issue has been open for over one year without activity and is not labeled as a bug. It has been labeled as stale to invite any further updates. If you can confirm whether the issue is still applicable to the latest version of Pants, appears in other contexts, or its priority has changed, please let us know. Please feel free to close this issue if it is no longer relevant.