cookiecutter-django
cookiecutter-django copied to clipboard
cookiecutter
Migrate to `uv` as package manager for the generated project
The project itself now uses uv and this is an attempt to bring uv to the generated project too.
- Python commands are executed through
uv runfor local development. In Docker the .venv python interpreter is added to the $PATH so packages can be used system wide. - The uv.lock file is empty on start because is is updated when
uv syncis executed. - For Heroku I added a
pre-committhat compiles a requirements.txt - Added a
traptotest_docker.shto cleanup after testing so the next time will no fail due to an existing postgres volume. - In test_docker.sh I added a step to test building the docker production image
Before we can merge this we need one last round to update all the versions to the latest ones in the requirement.txt files.
Ready for testing :tada:
cookiecutter https://github.com/foarsitter/cookiecutter-django --checkout uv-generated-project
Hi, there. I played around with using your fork to update some existing projects and eventually got them working but ended up reverting them to their production (pip) versions so that I could revisit when I had more time.
When working on an adapting an existing project, I got an error during the build, which was reproduced when I built a project from scratch using cookiecutter https://github.com/foarsitter/cookiecutter-django --checkout uv-generated-project:
[django python-build-stage 4/6] RUN --mount=type=cache,target=/root/.cache/uv --mount=type=bind,source=uv.lock,target=uv.lock --mount=type=bind,source=pyproject.toml,target=pyproject.toml uv sync --frozen --no-install-project --no-dev: 0.112 Using CPython 3.12.7 interpreter at: /usr/local/bin/python 0.112 Creating virtual environment at: .venv 0.113 error: Could not find root package
my-awesome-project'
I believe that it's caused by having the blank uv.lock. I thought since the local project had the host file rw that it would generate a usable uv.lock during the local build. Is there a way to get a usable version of uv.lock on the host filesystem without running uv sync on the host filesystem? Thanks for this--very cool!
@steveputman thanks looking into this!
During a build you cannot write to the host system.
Your interpreter is inside the container together with uv so you can run uv:
docker compose -f docker-compose.local.yml run django uv lock
Struggled a lot with this and it is not ideal. So hopefully someone has a better solution.
@foarsitter The Docker documentation was sort of vague on that, but now I know.
Your solution is better than mine (I had uv installed on my host system so just ran uv lock in the project directory.
Thanks for the help! Will let you know if anything else comes up.
I think you need to run uv sync on various start scripts (like celery), at least locally. Because you're mounting a folder to /app/.venv we actually don't have the installed virtualenv created in the base Dockerfile.
Of course we're also not sharing a virtualenv here, by my read/attempt to use this. It might be possible to just share a mount across all the services, but I'm having trouble getting that to work.
Great work! This project really need to have a better tool than the plain pip
The dependabot would need updating too. I dont know if it supports pyproject declarations
The dependabot would need updating too. I dont know if it supports pyproject declarations
I believe it does, I've found some discussions on the project, here's the link to the issue.
The dependabot would need updating too. I dont know if it supports pyproject declarations
This has been added now: https://github.com/dependabot/dependabot-core/issues/10478#issuecomment-2691330949
Tried it. I wonder if the requirements currently in requirements/production.txt should be moved to an optional dependencies section in pyproject.toml, so something like:
[project.optional-dependencies]
production = [
"gunicorn==23.0.0",
# ...
]
Currently they appear in the main project dependencies, so they will be installed in the development virtualenv.
They can then be installed by running uv sync --extra=production - or explicitly excluded with uv sync --no-extra=production
uv docs about optional dependencies.
(And thanks for doing the work - I look forward to it being merged :) )
@foobacca I see development as a superset of production. Everything from production needs to be replicated in development but production doesn't need a test framework for example.
When the sentry_sdk is added as an extra dependency I cannot use capture_exception for example because the package is missing. Another example is testing the gunicorn command. I cannot do such thing when developing when I do not have the package.
@foarsitter I see your point (though you can of course install the production dependencies if you want with uv sync --extra=production ).
I don't have a strong preference, but it does seem like a change from the master branch, where there is a separate requirements/production.txt file, suggesting that the maintainers liked to keep the production dependencies separate. So I wanted to make sure that change was explicit and intended.
I'll leave it to the maintainers. If they're happy with the change I have no objection. (And I'm quite capable of changing my project after running cookiecutter anyway :slightly_smiling_face: )
One other idea. I would expect the [project] and [dependency-groups] section to be the first and second sections in pyproject.toml - putting the overall project definition and dependencies at the top where it's easy to find them.
I can obviously edit it myself after running cookiecutter, so not a big deal if others prefer it at the bottom, but thought I'd mention it.
I don't have a strong preference, but it does seem like a change from the master branch, where there is a separate
requirements/production.txtfile, suggesting that the maintainers liked to keep the production dependencies separate. So I wanted to make sure that change was explicit and intended.
The production deps are installed locally on the master branch too, so not so much of change - see https://github.com/cookiecutter/cookiecutter-django/pull/4838 for more context
One other idea. I would expect the
[project]and[dependency-groups]section to be the first and second sections inpyproject.toml- putting the overall project definition and dependencies at the top where it's easy to find them.I can obviously edit it myself after running cookiecutter, so not a big deal if others prefer it at the bottom, but thought I'd mention it.
I agree. Dependencies should be at the top of the file
We use https://github.com/tox-dev/pyproject-fmt for dictating the order of appearance, so you don't need to worry about it :)
Rebased with main and added some documentation about the lock file. After the initial build uv lock needs to be executed to generate a lock file and, for convenience, the build should be executed again taking advantage of the lock file.
I think it is finally ready to be merged!
According to the cookiecutter docs .bat files are supported too, should that be a solution @luzfcb otherwise we should go with your version.
@luzfcb adopted your version because the .bat file didn't work out of the box. I think we are ready!
@foarsitter I am trying to test the branch for an existing project. I created a blank project with the branch and then tried to merged the differences with mine. I am not sure I completely understood what has happened to package requirements. I copied the packages I had added to local.txt and base.txt to pyproject.toml, but it still gives a ModuleNotFound error. Where should the list of needed packages be when using uv? Thanks!
Are you using docker? How did you add the requirements? Wich module cannot be found?
Are you using docker? How did you add the requirements? Wich module cannot be found?
Yes, using docker. I had added them to base.txt (they worked fine with pip) and when I saw the error I then added them to pyproject.toml too to no avail. The error is for django-money, but I have added all of these: "django-money==3.5.3", "html2text==2024.2.26", "mjml-python==1.3.5", "mercadopago==2.3.0", "stripe==12.0.0", "nameparser==1.1.3", "django-afip==13.2.0", "django-weasyprint==2.4.0",
Error is as follows: ModuleNotFoundError: No module named 'djmoney' (on the django container)
You can add all your dependencies with uv as we do in the post gen hooks: https://github.com/cookiecutter/cookiecutter-django/pull/5434/files#diff-8601710441b0dde909812d3f5a1d83245fec1f1bf94b6e510d1b72cbd2fb86c1R525
First you need to remove the extending (-r) of product.txt from local.txt. I think it would be wise to remove all the dependencies you added manually from pyproject.toml so things don't get mixed up.
Then you need to execute the following commands:
# project dependencies
docker compose -f docker-compose.local.yml add -r ./requirements/base.txt
docker compose -f docker-compose.local.yml add -r ./requirements/production.txt
# dev dependency group
docker compose -f docker-compose.local.yml add -r --dev ./requirements/production.txt
This will modify you pyproject.toml but doesn't alter your image. Therefore you need to build the image again with docker compose -f docker-compose.local.yml build django.
I think you didn't rebuild the image, and therefore, missing djmoney.
I get an error when trying to execute those commands (unknown docker command: "compose add")
Anyway, I had manually added the dependencies (including django-money) to pyproject.toml and did a build --no-cache and the error is still there. I actually did a test with --progress=plain to check whether django-money is every installed and no, it is never mentioned (although it sits there in pyproject.toml).__
I even tried running docker compose -f docker-compose.local.yml run django uv lock as you mention in an earlier comment before. uv.lock correctly references django-money, but the problem persists (uv doesn't seem to realize it has to install django-money). Not sure where it is taking the list of packages it needs to install from.
I might have missed merging something when bringing code over from the empty project I generated using this branch🤷🏻♂️
There is a mistake in the commands, it has to be uv add.
Do you have a .venv in your directory? If you have files in the directory it will override your .venv in your container.
Did you update the compose file according to this PR?
Cool, I managed to run uv add
Error is still there. I didn't have a .venv in my directory. django-money is inside the generated .venv/lib/python3.12/site-packages
I merged all of the changes in django's Dockerfile. I didn't find any particular changes in the actual compose file.
Do you have the following volumes listed in docker-compose.local.yml for the docker service and 100% sure there is no .venv folder? (rm -rf .venv)
volumes:
- /app/.venv
- .:/app:z
@acyment made some changes to the Dockerfile, can you try these?