fix(external_data): replay old external data before updating

[yajo]

Before this fix, when a template used local _external_data, it could never again be cleanly updated. It always gave conflicts.

This was happening because the old worker that did the replay always happened on an empty directory. Thus it never had external data available. Since external data can alter the render result (just like any other answer), the replay was never reliable.

Now, new data is saved in the answers file to be able to reliably replay including that context. Then, updates can work again as expected and result in meaningful diffs only when there should be.

BREAKING CHANGE: A new _dst_commit is added to the answers file, pointing to the commit just before the last template update. This is needed for reliable replays. This means that any copy or update operation will always produce diff when the subproject is git-tracked.

[sisp]

Yet another reason for https://github.com/copier-org/copier/issues/1170#issuecomment-2917409441, right? 😉 No more replay.

[yajo]

Indeed, that would solve this too.

[yajo]

I found out one way to workaround the issue.

Set up copier.yaml like this:

# Get external data from other template, or any other file
_external_data:
  parent_template: .copier-answers.yml

# Use a computed variable for the data you actually need from there, declared like this
version:
  default: "{{ _external_data.parent_template.version | d(last_version) }}"
  when: false
  type: float

Then set up {{ _copier_conf.answers_file }}.jinja like this:

# Changes here will be overwritten by Copier; do NOT edit manually
{{ _copier_answers|to_nice_yaml }}

# Smoother replays. Notice that `last_version` is NOT in the questionary
last_version: {{ version }}

Why this works?

1. When copying for the 1st time, version gets its value from the external data, which is available in that moment.
2. When updating, Copier will fail to get the external data because it does not have a notion about when the last copy was done. Thus it will fall back to using the value recorded in last_version.
3. After rendering, last_version will be recorded as the current version of that render. Thus, next renders will know the last value.

This works for me without introducing any backwards incompatibility, and the day #1170 gets fixed it will also fix this issue. Thus, closing.