icu4x icon indicating copy to clipboard operation
icu4x copied to clipboard

Regen tasks should only overwrite existing files if successful

Open Manishearth opened this issue 1 year ago • 2 comments

We have a bunch of regen-style tasks (testdata, diplomat-gen, fingerprint, ffi-coverage) where we generate a bunch of files on top of checked-in files (and typically, check the difference on CI)

They typically work by deleting the existing files, running the task, and then overwriting the file. This means that if the task fails, the files will be in a weird state that often needs to be reset from git. Furthermore, for testdata, git status is basically useless during the long-running testdata regeneration.

I think these tasks should run the generation to some temporary folder first, and only if the generation step succeeds, delete the old files and copy over new ones.

Manishearth avatar Aug 25 '22 21:08 Manishearth

Questions:

  1. Should we delete the temp directory if codegen failed, or not delete it for debugging purposes but leak disk space?
  2. How to make it work consistently on Windows and Unix?

sffc avatar Aug 26 '22 03:08 sffc

I think it's fine to delete if codgen failed, but we can have an env var disable it perhaps.

I assume duckscript has enough stuff to make this work cross platform.

Manishearth avatar Aug 26 '22 03:08 Manishearth