This is a proof-of-concept implementation for loading a new format for cookiecutter.json 😄

There are a number of features missing from this, but it should be sufficient to allow feedback on the specification. You can run the example via $ python cookiecutter/context.py. Nothing will be generated, but the prompting is done (hardcoded to verbose) and the resulting context as an OrderedDict is pprinted.

@audreyr @pydanny @michaeljoseph 👋

Please let me know your thoughts and I'll happily make changes to the spec.

Codecov Report

Merging #848 into master will decrease coverage by 15.24%. The diff coverage is 0%.

@@             Coverage Diff             @@
##           master     #848       +/-   ##
===========================================
- Coverage     100%   84.75%   -15.25%     
===========================================
  Files          16       18        +2     
  Lines         657      807      +150     
===========================================
+ Hits          657      684       +27     
- Misses          0      123      +123

Impacted Files	Coverage Δ
cookiecutter/context.py	0% <0%> (ø)
cookiecutter/config.py	100% <0%> (ø)	:arrow_up:
cookiecutter/hooks.py	100% <0%> (ø)	:arrow_up:
cookiecutter/cli.py	100% <0%> (ø)	:arrow_up:
cookiecutter/prompt.py	100% <0%> (ø)	:arrow_up:
cookiecutter/generate.py	100% <0%> (ø)	:arrow_up:
cookiecutter/__main__.py	100% <0%> (ø)	:arrow_up:
cookiecutter/environment.py	100% <0%> (ø)	:arrow_up:
cookiecutter/main.py	100% <0%> (ø)	:arrow_up:
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ceafa1d...6cc1831. Read the comment docs.

I would also like to hear from @freakboy3742 @jakubka and @zooba.

Please feel free to comment on this proposal with regards to help messages and dict values.

Thank you! 🙇

As I've already said in the chat, I really like this spec.

For everyone having a hard time reading the json from the diff, this is the spec in a gist: https://gist.github.com/hackebrot/fc96363e6c4166c6b1bf0a31245669ee

Could you maybe give an example on how exactly the skip_if thing works?

@jayfk If I understood correctly, in the example: https://gist.github.com/hackebrot/fc96363e6c4166c6b1bf0a31245669ee#file-cookiecutter-json-L61-L79 If the user responds negatively to the docs question, the question docs_tool will not be displayed

@hackebrot

1 - about skip_if feature, What is the value of cookiecutter.docs_tool if cookiecutter.docs has a False value?

2 - What happens if cookiecutter.docs_tool is used to define a name to a directory, but cookiecutter.docs has a False value?

3 - Which will be used to indicate the version of the cookiecutter.json file format? ( Thinking that in the future, there may be modifications in the cookiecutter.json file format. )

Anyway, after that first look, it sounds good to me.

ps, I have not tested the code. I'll have time to test it next Saturday.

3 - Which will be used to indicate the version of the cookiecutter.json file format? ( Thinking that in the future, there may be modifications in the cookiecutter.json file format. )

That's important. We should make the version a requirement for the new format.

Meta Information

There will be a number of fields that hold information about the template itself. As such they will not be contained in the jinja2 context, but allow us to implement features that will improve the user experience in the future.

Required Fields for a Template

• name - string: a name for the template

We can use this for dumping the JSON context for --replay. Currently we make a good guess based on the directory name of the cloned git repository, which is not great for local templates or relative paths. This could be something like an ID.

• cookiecutter_version - string: the version of cookiecutter

This either indicators the version of cookiecutter that this template requires or the version of the spec itself. Not entirely sure what's better in this case and if we want to separate them. Going forward this will allow us to exit early if the used cookiecutter CLI is not the latest one, but the template depends on a new built-in extension or new fields.

• variables - array of object: variables to prompt the user

The elements of this array represent a single variable, similarly to what you currently find in a cookiecutter.json file.

Variables

Required Fields for a Variable

• name - string: name for a variable in the jinja2 context

This is nothing different from what we have in the current cookiecutter.json as keys. These must not be templated!

• default - string, boolean or number: default value for the variable

Again this is what we already have as values. If a default is a string, we must assume it is templated, so we render it before promptig the user.

Optional Fields for a Variable

• type - string: the type for a variable

This defaults to string, which reflects the current behaviour (right now we cast every value to string, so we can render it). Having a type allows us not only to make use of click types for prompts, but we can also cast the values after they have been rendered.

• prompt - string: will be shown to the user when prompted for input

Currently we show variable [default]:, but a template author could provide a more friendly message allowing for a better user experience.

• prompt_user - boolean: simple flag for showing prompts or not

This can be used to hide prompts from a user if the template author wishes to use these fields but retrieve the information from somewhere else, for example the current year. This is currently supported with a hack by prepending a variable name with _.

• description - string: a text that describes what the variable means

We can show this if the users runs verbose mode, to make it even clearer for what a variable is used for and potentially indicate what the requirements for a field are, please see the example JSON in this PR.

• validation - string: regex pattern to check the input

This would allow us to have some additional checks for accepting user input. Think of PEP8 compliant names for Python modules. Rather than using a post_gen_project hook and abort generation, we could ask the user to try entering another value.

• choices - array of string, boolean or number: list of valid values for that variable

This is currently supported with lists in cookiecutter.json. However this field would be optional for a variable and is different from type in the sense that a choice will still be processed to have the specified type when stored to the context.

• skip_if - string: conditionals based on other fields

This one is a bit tricky. In it's current form it would be a string containing a jinja2 template. When prompting the user this is rendered and checked for equality against "True". This allows us to skip variables based on previously entered information.

In this particular case, we ask the user if they want to create documentation. If they do, we ask which tool for generation they would like to use. However if they decide to not have any docs, they will not be prompted for a tool. While this is a very simple use case, I case see how complex templates such as cookiecutter-django make use of this to add optional tools and even prompt the user for settings for these very tools.

After having played with context formats for a good while, I don't think nesting is not a good alternative. I'd much rather compose these relations rather then declaring them implicitely by nesting context variables.

Optional Fields for a Template

• description - string: a human readable description for the template

This can be used for user facing aspects, like a welcome message when running cookiecutter.

• version - string: a version identifier, ideally following SEMVER

This will help us generate helpful error messages.

• authors - array of string: maintainers for the template

Again this will help users in case they encouter issues. Currently users tend to raise issues on the cookiecutter project rather than the template. I would like to emphasize that template authors need to make sure that their templates work.

• license - string: license for the template code

The template itself is not runnable software, but contains source code. So I would argue that it should specify a license. Obviously this is not binding if the repository is missing a LICENSE file or w/e the license in question requires. We don't need this for MVP.

• keywords - array of string: simlilar to PyPI keywords

Providing keywords in a template makes it easier for tools, such as the new Cookiecutter Explorer in Visual Studio or Cibopath, to search for templates. Currently users need to go to the template repo and scan through the README or even the template code to see if a template uses certain frameworks.

(For Cibopath I use all of the words on the README as keywords, which is not great 🙈 )

• url - string: URL for the template project

We can use this to point users to the project if they encounter an error. This would certainly be optional.

I did some experiments with a standalone cookiecutter desktop gui application a while ago. The main idea is to have a little desktop app that can be used to browse available cookiecutter templates, fill out a form and render a template.

This is an image from a protoype:

In this context, it would make sense to add date and datetime to the spec. This would allow me to render a date or datetime select widget. Another useful thing would be an optional image field for a variable.

Is that something you would consider adding to the spec?

I see a lot of things to like about the proposed format. A few things that I'd like to see (over)specified in any official spec:

• cookiecutter_version should be a pip-like requirement, such that pip install cookiecutter{{cookiecutter_version}} would always get you a suitable version
  • this means >=2.0.0 is probably most common, but also makes >=2,!2.0.1 or >=2,<3 valid
• I'd rather have arbitrary code for validation
  • this would allow "option Y is invalid given previous selection for option X"
  • I think a post-value hook of some sort would work best: def hook(value_name, value, context) -> error_or_None
• I'd sort-of prefer the skip_if value to just be an eval() expression, but I can see the attraction of Jinja format here (I'm just not sure how valuable it is)
• Properties with an underscore at the start should be ignored (just like today)
  • for VS, we have a notion of a selector to help choose a value, but without defining the type of the value (e.g. a DB connection string is still a string, but the selector pops up specialized UI for databases)
  • I'm not sure that selector is a good cookiecutter-wide concept, but as long as we can have a _vs_selector field that won't conflict with anything, that's fine
• Personally, visible seems more obvious than prompt_user (which in itself seems like the opposite of skip_if)
  • GUI tools typically treat "visible" and "enabled" as separate states - I haven't tested this yet, but I think in VS we'd want to hide fields that can never be set, and disable fields based on skip_if so that users can see what options may be there if they change their mind
  • also, why not start the name with an underscore if you don't want it to appear? Would that disable other processing that I'm not aware of?

@huguesv and @brettcannon may have more suggestions, but overall I like the direction this is going.

I think this will solve all of my current wants. Using prompt_user to hide repetitive template expressions will be nice (I tried the _ prepend trick today but Cookiecutter didn't seem to want to render Jinja in those fields), and skip_if (when combined with some fancy prompt_user) will let me branch when people choose whether they want e.g. socket or WSGI by defining a private is_socket that uses an {% if cookiecutter.choice == "socket" %} block to set True or False.

Otherwise I'm down to only wanting to inject the Jinja context into the pre- and post-generation scripts as a wishlist. Great work, @hackebrot !

Brett's "current wants" are for brettcannon/python-azure-web-app-cookiecutter, since he left that out (that's also a good example of the _visual_studio extension property we're using to prettify rendering - this PR would make all of that redundant though, which we'll be happy to see).

I like it too. Here are some of my thoughts that haven't been brought up already.

It may be good to add examples of specifying _copy_without_render and _extensions in the new format.

Old format:

{
    "project_slug": "Foobar",
    "year": "{% now 'utc', '%Y' %}",
    "_extensions": ["jinja2_time.TimeExtension"],
    "_copy_without_render": [
        "*.html",
        "*not_rendered_dir",
        "rendered_dir/not_rendered_file.ini"
    ]
}

For specifying extensions, it would be good to have the pip requirements there in addition to imported module, like @zooba idea for cookiecutter itself. This way we have the ability to prompt or automatically install any missing extension.

I worry about localization more than most, and I wonder what that means for cookiecutter templates and Visual Studio. Most of Visual Studio is localized, and for cookiecutter GUI, does that mean localizing the prompts and descriptions in templates? What implications would this have on the json format? FYI I had filed my thoughts on what we could do a few weeks ago, based on the _visual_studio extended metadata we currently have: https://github.com/Microsoft/PTVS/issues/1758

This is a huge, HUGE change to Cookiecutter. It's a huge amount of work for the maintainers, forcing a 2.0 release. This is the easy part, but the real work is the ongoing maintenance burden. We've got a lot of experience with handling that in Cookiecutter.

Unless we have some sort of real sponsorship, be it financial compensation or an employer who will provide time, we should reconsider this change. Possibly reduce its scope or reject it.

Otherwise we the maintainers of Cookiecutter will be doing a gigantic amount of unpaid labor. And that unpaid labor is more often than not, going to be for large well-funded organizations. Having done enough unpaid labor for such organizations (besides a token amount from Google), I'm hesitant to do any more. I know @hackebrot really wants to work on this effort, but I am putting it on hold for now.

To identify this issue, I have put a "Requires Sponsorship" on this PR. If a well meaning organization wants to stand up and do the right thing, then I'll change it to a "Sponsored" PR label.

And then we'll move forward on this change.

I agree with what @pydanny said. While I can’t wait to implement and ship this exciting new feature, it is definitely going to increase the demand for maintenance, which inevitably eats up the team’s spare time. Unfortunately I can’t contribute to the project as part of my day job and neither Audrey, Danny, Michael or myself are paid to work on cookiecutter.

Danny is right, we need an organization to commit to supporting this project. Everything else is unfair to the maintainers. 😞

Totally appreciate that, the last thing you guys want is to be a critical piece of infrastructure run entirely on volunteer time.

Given my role at Microsoft, I can't actually offer any formal sponsorship (though I can - and will - raise the issue internally and see how far I get). We'll certainly have engineers around willing to help out on code/reviews/etc., but again, that's only an informal offer and if you guys need something really solid then that's probably not going to cut it.

In any case, as great as this change is, we already have a workaround for now which we couldn't change for our upcoming release anyway. IIRC we capped our cookiecutter dependency at <1.5 to avoid unexpected breakage, so there's really no rush from our side to get to a new version. But we have already shown it to a few people around here and they love the concept, so if it takes off (i.e. lots of users, lots of templates, etc.) it'll be easier to come back with real support. Knowing that you're willing to accept and work with that is great.

Will version 1.5 be released on PyPI before this PR is implemented?

@cheungnj I think so. I want to have a look at #850 though, before cutting a new release.

@hackebrot Thank you very much!

any news ?

No updates from me, I'm afraid. 😶

Hi, we are looking forward to these features in the cookie cutter.

currently, we found https://github.com/pydanny/cookiecutter-djangopackage to generate Django projects

We came across one requirement. If the user enters a list of environments [alpha, beta, dev, prod]. we would like as user certain inputs for each environment like which type of DB, [sqlite / postgresql], storage [s3 / local]. we would like to generate the environment specific settings or configuration apart from common files which we can generate from existing cookie cutter support.

settings
├── alpha
│   ├── alpha.py
│   ├── common.py
│   ├── db.py
│   ├── __init__.py
│   ├── sentry.py
│   └── storage.py
├── beta
│   ├── beta.py
│   ├── common.py
│   ├── db.py
│   ├── __init__.py
│   └── storage.py
└── __init__.py

@anush0247 this is an existing feature, and even if it didn't already exist, it does not belong to the discussion of this thread. Please open different feature requests in new issues.

Not sure how much "sponsorship" is needed, but here is a site for allowing us to contribute as I know my organization would be interested in helping make this a reality... https://opencollective.com/

@kevindiamond Could you open an issue about https://opencollective.com? This way we can move it to a separate discussion area.

Any updates?

still with complete lack of adequate funding to start this.

Any chance this will happen?

@adamaltmejd https://github.com/cookiecutter/cookiecutter/pull/1008 has a fork implementing a version of this if you are feeling adventurous. You can always try the cookiecutter.js 😜

Hi! I am having a hard time understanding how to skip variables. For instance, with cookiecutter 1.6.0 , can somebody write a small simple example just showing, if I have in my cookiecutter.json only two variables:

"prueba": [false, true], "parameter": "which is your parameter?"

How can I make parameter only appear if prueba is true?

Thanks a lot!

What is the current state of this? It includes many interesting features that I need

Let's do it 🚀