frontend
frontend copied to clipboard
home-assistant
Sync lokalise translations and disallow contributions
Checklist
- [X] I have updated to the latest available Home Assistant version.
- [X] I have cleared the cache of my browser.
- [X] I have tried a different browser to see if it is related to my browser.
Describe the issue you are experiencing
- Translations from lokalise are not synced with repository, so
- Development outside English is stunted, and
- Impossible to validate keys from core during linting
- No mechanisms in place to stop commits to
translations/*/*.json, so- Contributors get confused and waste time committing to these files, and
- Reviewers waste time catching and informing, or
- Accidentally merged and bug introduced
Describe the behavior you expected
After poking around all the scripts and workflows currently being ru and reading this blog post from lokalise, here's what I'd propose:
- [ ] Authenticate the lokalise GitHub app with the HA org and follow their instructions to setup automatic pull requests
- [ ] Protect those
lokalise-*branches and enable auto-merge on the generated PRs - [ ] Optionally, replace the
translations.ymlworkflow, and associated upload script using the lokalise CLI, with a simple web hook to their GitHub app. Saves us some code. - [x] Create a git pre-commit hook (installed via bootstrap) to fail for changes to
translationsdirectory (good defense, but won't catch commits without bootstrap or via web) - [ ] Modify
ci.ymlworkflow to fail on modifications totranslationsdirectory, except on PRs coming from lokalise
I can take care of the last 2, but admin access to the lokalise project and frontend repository are needed for the other changes.
Steps to reproduce the issue
n/a
What version of Home Assistant Core has the issue?
n/a
What was the last working version of Home Assistant Core?
No response
In which browser are you experiencing the issue with?
No response
Which operating system are you using to run this browser?
No response
State of relevant entities
No response
Problem-relevant frontend configuration
No response
Javascript errors shown in your browser console/inspector
No response
Additional information
No response
Instead of storing all translations locally, can we change it around such that it's easy for any developer to build a language locally with the latest lokalise data. It's a bit silly to just have constant PRs go into the project.
You need an API key to pull updated translations. And that key needs to be for a user with elevated access to the project.
Yeah I don't see a way around it either. All API calls need a token from the project.
I suppose you could move the syncing to a separate repository to keep the frontend quieter, and then make the translations directory a sub-module. Of course that has its own complications.
The translations are included in the nightly build assets. So we can probably adjust the script to use that, instead of mocking with a separate repo.
At the very least you'd have to hash it and provide that as an asset too to easily compare with your local copy to avoid repetitively downloading 30+ MB, extra time to unpack, etc.
And what if you change branches or tags? The keys probably need to change too, so you'd have to write code to take the date of the HEAD, grab the correct nightly, repeat.
I was with you at the beginning of this comment, and now I convinced myself it's not worth losing the benefits of having it in git.
It would have to be manually invoked. Most people are not developing languages locally. You develop RTL locally but not the actual strings.
That solution doesn't address being able to validate keys coming from core, but all I need for that is the English file so I'm sure I could figure out something else to cross that bridge.
I think it doesn't have to be manual if we just modify the nightly workflow to also create a tarball for just the translations and upload as another artifact. Then the file is only ~2.3MB. Any objection to that?
In the meantime I added the pre-commit hook to stop contributors from modifying those files.
I hate the fact that you can't link a PR without it closing automatically 😡
Okay now we'll have nightly translations: https://github.com/home-assistant/frontend/actions/runs/2889670763#artifacts
I wrote most of the script to download the translations using the REST API and ran into a snag. Apparently, you cannot download workflow artifacts unless you are logged in to GitHub. See https://github.com/actions/upload-artifact/issues/51.
I saw some users generating a nightly release to get around this and others using a third party app. Neither is ideal. Do we want to go that route or just require developers to authenticate until GitHub changes course?
requiring to set GITHUB_TOKEN should be fine, that can even be a vscode task Like this one https://github.com/home-assistant/supervisor/blob/main/.vscode/tasks.json#L32-L45
But that requires input every time the task is run, correct? My goal would be to just have it be part of the build process, so the token would have to be stored.
Even if a task is added you can call the script directly. As long as it's documented it does not matter how it's done.
@balloob I've got this working nicely using device flow authentication to GitHub. Before I submit the PR, I transferred ownership of the GitHub app I've been testing with to the HA org, so you need to accept it and then install it to the frontend repo.
Let me know if you have questions.
Why does this needs to be a GitHub app, and not only an OAuth app? This is only used to authenticate the user right?
It can be either one and I've tested with both. If it's an OAuth app, the download API call won't work without the public_repo scope, which gives the token read/write on all public repositories. The GitHub app needs no permissions at all, so I favored it just for security.
For this purpose, they both are using OAuth2 under the hood to authenticate the user. See https://github.com/octokit/auth-oauth-device.js.
You mean needing the public_repo scope? It's not documented well at all and I've provided feedback in several places: https://docs.github.com/en/rest/actions/artifacts#download-an-artifact.
The general GitHub behavior of needing to be logged in to download artifacts is also not documented anywhere I could find, but plenty of people annoyed by it.
Instead of storing all translations locally, can we change it around such that it's easy for any developer to build a language locally with the latest lokalise data. It's a bit silly to just have constant PRs go into the project.
Not that I'm excited to point this out after already putting in the work to write the script, but isn't this what the core, iOS, and android repositories already do? I was sniffing around for project IDs and noticed they are all pulling translations from Lokalise nightly and committing them.
You're right and not sure why I wrote that 🤔 I guess the issue here is validation of keys + preventing people making changes, which is what is the new script is solving 👍
FYI this issue has affected 3 PRs just in the last month. For some reason the mentions aren't showing up in the thread.