Fix i18ntokens generation

[tkajtoch]

Summary

This fixes the generation of i18ntokens.json and i18ntokens_changelog.json files used to determine whether there have been any changes to i18n token names or default messages since the last release.

This PR also includes a few cleanup items to make EUI build scripts easier to understand.

i18ntokens files need to be created on build, but committed only during release to enable testing in Kibana.

QA

1. Checkout this branch locally
2. Run yarn workspace @elastic/eui run build to build EUI and confirm there are changes to packages/eui/i18ntokens.json and packages/eui/i18ntokens_changelog.json files
3. Run yarn workspace @elastic/eui run prerelease to run the prerelease script executed during the release process by release CLI and confirm the changes in packages/eui/i18ntokens.json and packages/eui/i18ntokens_changelog.json are committed as chore(eui): update i18ntokens
4. Drop the created commit

[tkajtoch]

I spent quite some time fixing the bug pointed out above by @acstll. I'm back from PTO and will push the changes I had in progress shortly.

[tkajtoch]

@acstll I ended up having to refactor the whole i18ntokens logic and spent lots of time understanding what's even happening there. The code is very similar to the original one but simplified in places where that was straightforward to do. The bug you reported is finally fixed!

FYI the i18ntokens.json changes heavily because the new logic, specifically the globIterator traverses files in a different order. The keys are the same but sorted a different way.

[elasticmachine]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: adeec94094dfaefb8386a16476af8f615bcde63a
• Documentation website

History

• :broken_heart: Build #656 failed 7e0e468cef8e1aa6313876a20a9706b655c016e5
• :broken_heart: Build #653 failed 72fa6245e9b4a5dfc18fab303ac8d26b0e3d1ead
• :green_heart: Build #294 succeeded 0c205b595bf5ee34f621d372de4b774a00684492

cc @tkajtoch

[tkajtoch]

@acstll I had to fix one more thing with the usage of glob.sync - it turned out to not copy the files as it was supposed to. Could you take one more look at the latest commit before we merge it?

[tkajtoch]

[nit/suggestion] have you considered adding // @ts-check to the top of the new .mjs files?

I haven't 🤔 I'm happy to explore it if you don't mind adding it separately. I'd like to read more about it first

[elasticmachine]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: adeec94094dfaefb8386a16476af8f615bcde63a

History

• :green_heart: Build #4289 succeeded 7e0e468cef8e1aa6313876a20a9706b655c016e5
• :green_heart: Build #4023 succeeded 0c205b595bf5ee34f621d372de4b774a00684492

cc @tkajtoch

[acstll]

@acstll I had to fix one more thing with the usage of glob.sync - it turned out to not copy the files as it was supposed to. Could you take one more look at the latest commit before we merge it?

I built main, the previous glob.sync change, and latest and in fact there was something wrong previously, lots of files were missing… (how did you notice? 😅)

🟢 I can confirm the latest commit generates the right amount of files (is in main).

---

[nit/suggestion] have you considered adding // @ts-check to the top of the new .mjs files?

I haven't 🤔 I'm happy to explore it if you don't mind adding it separately. I'd like to read more about it first

definitely, not at all 👍

[tkajtoch]

how did you notice? 😅

I noticed an error while trying to run the website dev server about a missing JSON file :D