Add `_updatable: true` to `copier.yml` instead of having to create the answers file by hand

[heavelock]

Describe the problem I created my first template with {{_copier_conf.answers_file}}.jinja file that was completely empty. I ran copier package_template new_package which created a new project. Then I re-ran the same command copier package_template new_package which lead copier to crash. File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/subproject.py", line 66, in last_answers for key, value in self._raw_answers.items() AttributeError: 'NoneType' object has no attribute 'items'

To Reproduce

1. Have a template with existing yet empty file \{\{_copier_conf.answers_file\}\}.jinja. Have at least 1 question
2. Run copier package_template new_package. Answer anything
3. Run again copier package_template new_package.
4. Copier crashes

If the template file for the answers contains jinja2 template, everything works properly.

# Changes here will be overwritten by Copier
{{_copier_answers|to_nice_yaml}}%

Logs

Full exception stack.

✦ ❯ copier sisyphus_package_template aa
Traceback (most recent call last):
  File "/Users/qsbt/.local/bin/copier", line 8, in <module>
    sys.exit(CopierApp.run())
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/plumbum/cli/application.py", line 639, in run
    inst, retcode = subapp.run(argv, exit=False)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/plumbum/cli/application.py", line 634, in run
    retcode = inst.main(*tailargs)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/cli.py", line 71, in _wrapper
    return method(*args, **kwargs)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/cli.py", line 294, in main
    self.parent._worker(
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/main.py", line 620, in run_copy
    src_abspath = self.template_copy_root
  File "/Users/qsbt/.asdf/installs/python/3.9.1/lib/python3.9/functools.py", line 969, in __get__
    val = self.func(instance)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/main.py", line 593, in template_copy_root
    subdir = self._render_string(self.template.subdirectory) or ""
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/main.py", line 567, in _render_string
    return tpl.render(**self._render_context())
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/main.py", line 231, in _render_context
    **self.answers.combined,
  File "/Users/qsbt/.asdf/installs/python/3.9.1/lib/python3.9/functools.py", line 969, in __get__
    val = self.func(instance)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/main.py", line 351, in answers
    last=self.subproject.last_answers,
  File "/Users/qsbt/.asdf/installs/python/3.9.1/lib/python3.9/functools.py", line 969, in __get__
    val = self.func(instance)
  File "/Users/qsbt/.local/pipx/venvs/copier/lib/python3.9/site-packages/copier/subproject.py", line 66, in last_answers
    for key, value in self._raw_answers.items()
AttributeError: 'NoneType' object has no attribute 'items'

Expected behavior I would expect copier to check if it can render template for the .copier-answer.yml and complain if it cannot since it's a vital file for the template.

Maybe even create a template for a template that would be shipped with copier? copier new_template that would create all expected files for a template?

Environment

• OS: macos 12.6.3
• Copier version: 7.0.1
• Python version: 3.9.1
• Installation method: pipx+pypi

[yajo]

Hi there!

I understand the issue, thanks for posting it!

To fix your current problem, you'll have to remove the broken answers file, commit and copy again.

I won't reject the fix if you want to contribute it, but I don't plan to add it. After all, I think it's normal that, if you don't follow the docs regarding how to create the answers file, Copier crashes. 😅

Maybe even create a template for a template that would be shipped with copier? copier new_template that would create all expected files for a template?

Hey such a good idea!

I'll close this issue and open a new discussion about this idea: https://github.com/copier-org/copier/discussions/1011

[sisp]

I wonder whether we actually need to require a Copier template to contain a file {{ _copier_conf.answers_file }}.jinja. It's always the same anyway, so why burden the template creator with it? Even when using a dynamic subdirectory for managing multiple templates in the same repo where each sub-template has a {{ _copier_conf.answers_file }}.jinja file (whose contents are completely identical by the way), I think this file is not needed. Copier's default is $dst/.copier-answers.yml which can be overridden via the copier.yml setting _answers_file or via the CLI flag --answers-file or via the API argument answers_file, and {{ _copier_conf.answers_file }}.jinja is rendered to whatever the path and filename is in the end.

So why not omit the {{ _copier_conf.answers_file }}.jinja entirely and simply create this file when generating a project?

WDYT, @yajo and @heavelock?

/cc @pawamoy

[heavelock]

From my perspective of a first time user, it was a bit confusing that there was an answer file that I had to prefill with a required content. Of course, I admit this is my fault because it is written in the documentation and I should have followed it better. However, I completely agree with you @sisp that pre-existence of this file is not really necessary.

[yajo]

The file is needed because it is a way to indicate that you support updates. Without it, your template becomes bootstrap-only, just like cookiecutter or yeoman. It also helps on advanced use cases like meta-templates.

I agree it could be replaced by a _updatable: true instruction. The answers file was just the MVP back then when I invented it. 😅

[sisp]

I think neither _updatable: true nor the answers file template are needed, instead a new CLI flag could be introduced which is less intrusive IMO.

The different ways of generating a project would look like this:

• copier $src $dst generates a project with the default answers file .copier-answers.yml because this is backwards compatible and a reasonable default as most projects will want updating support. Also, _answers_file: .custom-copier-answers.yml can be specified in copier.yml as before.
• copier --answers-file .custom-copier-answers.yml $src $dst generates a project with a custom answers file name (and possibly path). This behavior is the same as it is now, so no change.
• copier --no-answers-file $src $dst generates a project without an answers file, so it's a bootstrap-only project. --answers-file and --no-answers-file would be mutually exclusive flags. Or perhaps copier --answers-file "" $src $dst could be an alternative without introducing a new flag. _answers_file: "" in copier.yml would also omit the answers file unless overridden by the --answers-file flag.

Project updates would look like this:

• copier update $dst when the default .copier-answers.yml file is used or a different one is specified in copier.yml via _answers_file: ....
• copier --answers-file .custom-copier-answers.yml update $dst when a custom answers file is used via CLI flag.
• copier update $dst raises an exception when the default answers file or the one specified in copier.yml via _answers_file: ... is found.
• copier --answers-file .custom-copier-answers.yml update $dst raises an exception when the answers file is not found.

WDYT?

[yajo]

I like the fact that updates support is opt-in. I myself have updatable and non-updatable templates. I wouldn't like to loose that. Supporting updates adds load on the template maintenance, so I think it's better to start from simple to complex.

Reusing the answers file option is IMHO less explicit. That would make updates opt-out instead. I don't see many benefits TBH. 🤔

[sisp]

Opt-in vs. opt-out is a fair point. I'm not opinionated at all. The current behavior is fine for me.

[sisp]

I still find the {{ _copier_conf.answers_file }}.jinja file redundant and have thought of a backwards compatible opt-in approach to remove it:

• If a template has the {{ _copier_conf.answers_file }}.jinja file, Copier behaves as usual. That is, the template supports updating and the _answers_file setting in copier.yml and the -a, --answers-file CLI flag allow overriding the default location and filename.
• If a template does not have a {{ _copier_conf.answers_file }}.jinja file,
  • then it does not support updating, unless
  • _answers_file in copier.yml is set to true or to a file path relative to the generated project's root directory. This may be overridden by the -a, --answers-file CLI flag.

This approach is backwards compatible as we would still support the {{ _copier_conf.answers_file }}.jinja file and respect it. But in its absence, a template could still support updating by setting the _answers_file setting in copier.yml where true is a shortcut for implicitly using the default filename .copier-answers.yml. Setting it to false would be identical to omitting this setting, and without the presence of {{ _copier_conf.answers_file }}.jinja file the template would not support updating. When _answers_file was not set (or was false), passing -a, --answers-file would be disallowed because a user cannot make a template support updating (by enabling an answers file) that doesn't actually support updating (because there's typically more to updating than just the answers file).

To illustrate the compatibility and behavior of the different options, this table might help:

Answers file path in template	_answers_file value	-a, --answers-file value	Valid?	Supports updating?	Answers file path in project
			✅	❌
		anything	❌
	false		✅	❌
	false	anything	❌
	true		✅	✅	$dst/.copier-answers.yml
	true	.custom-answers.yml	✅	✅	$dst/.custom-answers.yml
	.config/copier-answers.yml		✅	✅	$dst/.config/copier-answers.yml
	.config/copier-answers.yml	.custom-answers.yml	✅	✅	$dst/.custom-answers.yml
$src/{{ _copier_conf.answers_file }}.jinja			✅	✅	$dst/.copier-answers.yml
$src/{{ _copier_conf.answers_file }}.jinja	.config/copier-answers.yml		✅	✅	$dst/.config/copier-answers.yml
$src/{{ _copier_conf.answers_file }}.jinja		.custom-answers.yml	✅	✅	$dst/.custom-answers.yml
$src/{{ _copier_conf.answers_file }}.jinja	.config/copier-answers.yml	.custom-answers.yml	✅	✅	$dst/.custom-answers.yml

[sisp]

See @yajo's reply:

I think I prefer the UX proposed in #983 (comment), so _answers_file stays as the way to provide an alt name if you want, and _updatable stays as the new replacement for adding the templated file.

— https://github.com/copier-org/copier/issues/1257#issuecomment-1646138419

[sisp]

Don't you think _answers_file plus _updatable is redundant? When {{ _copier_conf.answers_file }}.jinja does not exist, specifying _answers_file would require _updatable: true while _answers_file already implies updatability, so _updatable: true is actually implied. And specifying _answers_file with a file path without _updatable: true wouldn't make sense. So I thought to reuse _answers_file by allowing _answers_file: true to use Copier's default answers file name/path. :thinking:

[yajo]

Our current design is that each option has 3 possible sources: API, CLI and copier.yml. Some of them are disabled from some of those sources because of reasons, and it's documented appropriately.

answers_file is one of those settings supported explicitly from all those 3 sources. However, updatable should be supported only inside copier.yml.

Also, passing answers_file from API as a bool wouldn't make sense, and from CLI it would be impossible. Still, we'd have to support that.

Also, what if _answers_file: "{{ some_answer }}" renders to true? Should we treat it as True or as "true"? If we turn this option into a mixed bool/str type, we won't have a clear answer.

So, to avoid ambiguities, I think a new, explicit, bool-only option is better.

specifying _answers_file with a file path without _updatable: true wouldn't make sense.

It would make sense if you have a {{ _copier_conf.answers_file }}.jinja file.

[sisp]

Thanks for your detailed reply. You're arguments are convincing, let's stick with _updatable. :+1:

[lhupfeldt]

I agree that it would be nice to get rid of the template, which is confusing. I really think it should be opt-out if you don't want to allow updates. Updates is probably the strongest feature of copier. Many users would be surprised when trying to update and it fails because they did not set _updatable: true. I definitely did when I created the first template and put the answers template file in the top directory instead of my template subdir.

[yajo]

That's an interesting PoV. It'd be a breaking change if we default to true, but it makes sense indeed.