openverse
openverse copied to clipboard
Cache translations from GlotPress for 24 hours in CI instead of pulling on every single CI run
Current Situation
It's possible for our CI/CD suite to run enough times for us to get rate limited when downloading translations from GlotPress. See this run, for example:
https://github.com/WordPress/openverse/actions/runs/9937426887/job/27447855989?pr=4585#step:4:1007
We pull translations from GlotPress every time we build the frontend image, which happens any time frontend changes occur. We also pull it whenever storybook is built (because storybook build runs the build
script, and prebuild
runs i18n
which fetches translations). Storybook is built for documentation site changes as well as relevant frontend changes.
For certain change sets, we could conceivably fetch translations from GlotPress four times:
- First to build the frontend image
- Next to check storybook build
- Next to run storybook playwright tests
- Next to build the documentation site
Suggested Improvement
The easiest change to make would be to remove translation getting from storybook. We do not currently have any means of changing the language used to render components in storybook, as far as I can tell. The RTL/LTR toggle just changes the direction, but leaves the English language text—see this story, for example, which uses text from the translation files rather than hard-coded text from Storybook variables; there is no way to change the language! Refer to https://github.com/WordPress/openverse/issues/4621 the implementation of this "first step" which should unquestionably be implemented. This issue (4614), however, is for talking about caching the translations to reduce the overall necessary GlotPress interactions to a baseline of 1-per-day. Stopping Storybook unnecessarily pulling translations still pulls translations every single time the frontend builds in CI.
Realistically, there's no reason to pull translations so often. If we pulled translations in CI once every 24 hours (or a shorter set interval) and cached it in a shared artefact instead, it would speed up CI (even if only marginally), and prevent this problem from happening altogether, regardless of how active our repository is. A 24 hours SLA of translation changes is not unreasonable, and if there was a problem where we needed to pull translations right away and push a new build, we could easily manually dispatch the translation pull workflow and build a new frontend image.
Benefit
Prevent rate limiting to GlotPress and slightly speed up frontend (and storybook, so therefore also documentation site) build times in CI.