deps icon indicating copy to clipboard operation
deps copied to clipboard

Add draft for extended project template

Open knyghty opened this issue 1 year ago • 26 comments
trafficstars

This is a draft draft until I get around to writing the reference implementation. It's mostly copying files so shouldn't take too long, just need the motivation :)

Happy to have early eyeballs on it though.

knyghty avatar Oct 26 '24 14:10 knyghty

Is this the appropriate place to have open-ended discussion about this topic? Or should that be reserved for the forum?

Big picture here's my two cents:

  1. I love the idea of this initiative, though I think there will be widespread disagreement about what exactly should go into a new template.
  2. I think this is the most important part of the DEP, "it will set a precedent for adding new templates into Django" and would advocate for a strong focus in making it possible to more easily use different templates. So much so that I wonder whether "single-command startproject templates with 2-3 official reference implementations" might be a more suitable DEP (or at least easier to get consensus on). Yes, I know you can already pass a folder or URL to startproject / startapp and I still think we can do better¸e.g. by having a list of options for people to compare in the docs, or potentially by making it easier for template authors to make them more discoverable and configurable.

czue avatar Oct 28 '24 06:10 czue

Hiya. Fly-by comment. Thinking actively on this, rather than not responding. 😅

I think here is a fine place to discuss, at least to get to good draft status.

The motivation section is key. (… 🤔 TBC)

The current project isn't bad — it's just too many files for a project just kicking off. I think it's as we reach that level of complexity and greater that opinions really start to ramify, so keeping the existing template, and using that as the outer limit of what Django itself will provide is perhaps helpful. That leaves space for a couple of simpler templates.

Then, pondering the startproject command. (… 🤔 TBC)

carltongibson avatar Oct 29 '24 07:10 carltongibson

One thing I thought of, though I think it's a bad idea, is related to Cory's idea (maybe exactly the same), is that to get whatever new thing to be the default without making it the default could be to have a new command, e.g. django-admin new that would ask some questions and choose a template based on that.

I'm not sure if it's worth putting in the DEP, because it would either require a package to handle the "UI", a good amount of code I'm unsure how to write, or a very basic interface (such as when choosing options when creating migrations). Leaning towards the last of those three things. And of course having --noinput to use whatever we choose as the new default. Then startproject can stay around as is, or be eventually removed in favour of new.

It would change the scope of the DEP though, maybe not in a bad way, but still, wasn't what anyone had in mind I think.

Just random ideas :)

knyghty avatar Oct 29 '24 07:10 knyghty

To expand slightly, the question could be as simple as:

What type of project do you want to create?

  1. A standard project (default)
  2. A fancy pants project with frontend things added for you (???)
  3. A single file quickstart project for prototyping
  4. A third party app
  5. A classic project

Where a classic project would be the current template.

More things to maintain, though. And going this road leads to questions: should we add a pyproject.toml, how much configuration should we have, how opinionated should we be, etc. I would err on the side of doing as little as possible, though pyproject.toml feels standard enough to add for the user by now. But I also don't want to get lost in the rabbit hole of .editorconfig, .gitignore and other things that I always use, but are probably too opinionated (some people are probably still using mercurial, for example).

knyghty avatar Oct 29 '24 07:10 knyghty

The motivation section is key. (… 🤔 TBC)

I agree here and I think my motivation section is too light. It doesn't feel very compelling to me. Which is strange because a lot of people have strong opinions on why we need to change it, including myself. Open to ideas, but I also plan to properly sit with this and think it over.

knyghty avatar Oct 29 '24 07:10 knyghty

… properly sit…

Yes, me too. That’s why just the initial comment. 🤹

carltongibson avatar Oct 29 '24 07:10 carltongibson

Agree the motivation is key. If the motivation is solely as-written (improving the tutorial experience) then my comment/position doesn't really apply. My motivation is improving the speed and choices for people who want build things with Django.

wasn't what anyone had in mind I think.

Who is the "anyone" referenced here? When i read through the linked forum posts I don't really see any obvious consensus, so just wondering if there's a backchannel motivation for this specific change (or a discussion I missed elsewhere). I don't want to derail an initiative to make progress because it's not enough progress if I'm bringing in requirements that feel out-of-scope.

If we are trying to broaden the scope I would have more to say (and think about!) but will hold off for the time being...

czue avatar Oct 29 '24 08:10 czue

Who is the "anyone" referenced here

Anyone. Nothing backchannel. But I didn't see this mentioned anywhere. So we have a consensus (I think) to do something, but the approaches vary, and the one I just mentioned hasn't been talked about that I know of.

But this broadening of the scope would allow us to do more (possibly: enough) without breaking backwards compatibility. But I do need to go think about it.

knyghty avatar Oct 29 '24 15:10 knyghty

I really like this suggestion. To not do to much in one, it sounds sensible to only add the required templates and files, and not cram in other nice-to-have files. A project.toml may be just simple enough though.

Since it will also potentially draw in many new dependencies, maybe the fancy-pants option should be dropped, and replaced with a (as last option) "Paste a link to a project template" as startproject supports now, but is somewhat hidden?

(Since i just popped up out of nowhere - i picked up this PR from the Django Discord, also relevant: https://github.com/Kagee/django-single-folder-project-template )

Kagee avatar Oct 29 '24 18:10 Kagee

Since there was a lot of positive feedback on the new command I have included this in the proposal.

I've also fleshed out the motivation section for different user types.

I will try to write up the reference implementation during this weekend.

knyghty avatar Nov 02 '24 10:11 knyghty

I suggest making it clearer in the text/wording that the new default project will use the "merged project/core app layout", not just a classic project and a classic preinstalled core app. While it is "explained" using the tree view, i feel it is not clear from just reading the text.

Kagee avatar Nov 02 '24 14:11 Kagee

@Kagee makes sense, I updated the wording.

knyghty avatar Nov 02 '24 15:11 knyghty

Have you considered referencing the different templates using slug-names, so they could be chosen when batch-processing, possibly based on the folder-names for the templates to be used? merged-project-app, single-file, third-party-app, classic-project-app?

Kagee avatar Nov 02 '24 15:11 Kagee

I added a link to a reference implementation. I didn't test that the project templates actually work, but I think it gets the idea across.

One thing I noticed, startapp has a context variable camel_case_app_name used for the AppConfig class name. We'd likely need to add something like camel_case_proejct_name to startproject as well, to get this working there.

knyghty avatar Nov 09 '24 10:11 knyghty

Afaik camel_case_project_name already works? Ref myself https://github.com/Kagee/django-single-folder-project-template/blob/main/project_name/apps.py-tpl#L4

Kagee avatar Nov 09 '24 10:11 Kagee

Indeed. Not very well documented :)

I've changed that now in the implementation. Thanks for the heads up.

knyghty avatar Nov 09 '24 10:11 knyghty

will there be any functionality for 3rd party templates to use django-admin new? that is something I would hope we could preserve from startproject. i also think it would be nice if it were slightly more flexible than the current startproject customization.

I think to foster an ecosystem of useful 3rd-party templates the ideal syntax should be something like:

django-admin new <project-name> --template-url https://github.com/owner/repo

czue avatar Nov 20 '24 14:11 czue

@czue I'm not planning this as a replacement for startproject, which already does this. You can still use a tempalte URL by pasting it instead of a number, this is mentioned in the prompt of the reference implementation when you use it: https://github.com/knyghty/django-new/blob/main/django_new/management/commands/new.py#L26

knyghty avatar Nov 20 '24 15:11 knyghty

@carltongibson, since I have you down as shepherd - I've advertised this everywhere I can think of and only gotten positive responses and mostly crickets here.

What do you think? Should we merge? Does it need more detail? Do I need to drum up support? Wait to see how the SC organises itself? Something else?

knyghty avatar Dec 13 '24 20:12 knyghty

@knyghty I think we should let the SC election unfold, and circle back to this in January. (We can't advance it without a functioning SC in any case.)

carltongibson avatar Dec 14 '24 06:12 carltongibson

Just throwing one thing in there, should the command be called new or init?

Just from the top of my head:

  • git init
  • uv init

hvdklauw avatar Dec 24 '24 19:12 hvdklauw

FWIW cargo has both new and init -- where new is the one in normal use, and init is for creating a new package in an existing directory (in line with git init). poetry also has new and init behaving similarly (new requires a directory path).

Current startproject expects a path for an existing directory (or uses the current), so it's more like these inits; I'm not quite sure which of these should be taken as a guide, but I'm noting that the goal is to be more accessible and predictable to people new to Django, and unlike startproject, both poetry and cargo had their CLI designed in the last decade.

shaib avatar Dec 24 '24 19:12 shaib

Hey @knyghty -- I'd like to progress this during the current cycle. I'll raise it with the SC soon(ish). If you want to IDK go over anything please do, otherwise I'll have a proper read again and feedback soon(ish).

(Sorry for the ishes 😜)

carltongibson avatar Feb 04 '25 19:02 carltongibson

The DEP itself is good to review. The only remaining TODOs (and perhaps aren't needed before judging this) are for the linked reference implementation:

  • Add a pyproejct.toml to the "default" proejct
  • Add a template for a third party app

I do hope to get to these soon, but not sure if they're necessary to have fully in place or not (suspect not).

knyghty avatar Feb 05 '25 13:02 knyghty

My comment may not be directly related to the DEP and out of scope, but I'd like to mention it because implementing this DEP maybe could impact the code currently used by startapp / startproject.

In cookiecutter-django (If you haven't heard of it, it's basically a highly opinionated django project template), we place all our Django apps within a namespace. This approach keeps the project better organized in the long run (in our opinion) and simplifies the process of packaging and distributing the apps (on our internal pypi like server)

Therefore, when creating a new app in a project based on the cookiecutter-django template, we need to:

1 - create the <name-of-the-app> app with python manage.py startapp 2 - move <name-of-the-app> directory to <project_slug> directory 3 - edit <project_slug>/<name-of-the-app>/apps.py and change name = "<name-of-the-app>" to name = "<project_slug>.<name-of-the-app>" 4 - add "<project_slug>.<name-of-the-app>.apps.<NameOfTheAppConfigClass>", on your LOCAL_APPS on config/settings/base.py

In the past, I tried to extend the startapp code to create a new startapp like management command that supported creating the app in a namespace, but at that time, most of the startapp logic was contained within a single method and this would have been difficult to maintain in the long run.

In short, I basically wanted to run

django-admin startapp namespace.app_name

and have django create the directory structure and app respecting the namespace, that is:

namespace/
    __init__.py
    app_name/
        __init__.py
        admin.py
        apps.py
        models.py
        migrations/
               __init__.py
        tests.py
        views.py

and if the namespace already exists but the app_name not, simple create the app_name inside the namespace

The question is: Are there any plans to make startproject code and/or startapp code a bit more reusable and extensible by third parties?

Or perhaps simply implement namespace support in startapp?

luzfcb avatar Feb 05 '25 16:02 luzfcb

Or perhaps simply implement namespace support in startapp?

This part could arguably be in scope, because the new "default" startproject template proposed here comes with (at least) a bit of an implication that nesting apps under some global project_name namespace is a good idea, and I personally am a proponent of this.

I haven't given any thought to making startproject and startapp more extensible, and I would consider it out of scope for this DEP.

knyghty avatar Feb 05 '25 18:02 knyghty

The Steering Council has discussed this DEP and we're happy to approve adding the django new command and approve in principle the various proposed templates, which can be added individually as and when they're ready.

LilyFirefly avatar Oct 16 '25 18:10 LilyFirefly