nvaccess
discuss:Compile NVDA with GitHub Actions
hi The appveyor currently used by NVDA cannot compile multithreaded when compiling NVDA
Is your feature request related to a problem? Please describe.
If possible, we can compile NVDA with GitHub Actions
Describe the solution you'd like
help docs GitHub Actions docs and Events that trigger workflows - GitHub
Describe alternatives you've considered
none
Additional context
The following is a list of images supported by GitHub Actions: ubuntu-latest ubuntu-18.04 ubuntu-16.04 windows-latest windows-2019 windows-2016 macOS-latest macOS-10.14 thanks
I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of https://github.com/actions/upload-artifact/issues/3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see https://github.com/actions/upload-artifact/issues/11.
Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired.
I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.
Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired. Thank you for your instructions
One of my back burner projects has been to convert the Appveyor script to something more generic. The goal being so that people who fork the main repo, do not actually need a source compilation environment to test their own PRs and private changes, or otherwise build from source for their own purposes.
I haven't gotten far with it because of lack of time, but it seems to me that if we could get this working as an Appveyor alternative, it would make that idea ultimately a bit more straight forward.
Edit: implemented in #16221, #16293.
I agree that appveyor is not very ideal, however, our current appveyor script uses pretty specific functionality, also related to tests. It will take some time before actions is mature enough.
I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.
Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired.
Has the above problem been solved by github actions?
appveyor has started to limit the number of downloads to. {"message":"Artifacts download limit (1024 MB/day) exceeded."} test method is as follows: first open this address Once opened, the above prompt will appear {"message":"Artifacts download limit (1024 MB/day) exceeded."}
As actions is much more mature now and a switch to actions could be a possible way to avoid #13281, I"ll reopen this until a solution for #13281 has been chosen.
never mind, this is still a showstopper for us:
During a workflow run, files are uploaded and downloaded individually using the upload-artifact and download-artifact actions. However, when a workflow run finishes and an artifact is downloaded from either the UI or through the download api, a zip is dynamically created with all the file contents that were uploaded. There is currently no way to download artifacts after a workflow run finishes in a format other than a zip or to download artifact contents individually.
I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.
Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired. hi Maybe Creepios/sftp-action: GitHub Action to upload files/directories via ssh This problem can be solved
@LeonarddeR Your show-stopper seems no longer to be the case. Can you confirm that your concerns are satisfied?
After https://github.com/nvaccess/nvda/issues/13281, snapshots are no longer directly downloaded from appveyor. This means that the fact that github only offers zip files is less relevant, though it is still a bit of a complicating factor when downloading artifacts from pull requests. That said, I think there's enough potential to reopen this.
Hi, just to give an update from NV Access. Moving to GitHub actions is on our roadmap due to many issues with AppVeyor:
- AppVeyor instances appear to be getting slower, slowing the build times
- We can only perform one build at a time
- GitHub actions allows us to split up the CI/CD pipeline (e.g. the lint check can run independently of the build)
- GitHub actions are easier for forks to use
- being in one platform (i.e. GitHub) is generally simpler
@seanbudd wrote:
- AppVeyor instances appear to be getting slower, slowing the build times
We have long had the ability to make those significantly faster, by caching already compiled components and deploying them to the run environment through AppVeyor's caching options.
I often wondered why NV Access didn't, especially for the slow-to-compile C components, and clonable items that hadn't been updated, but assumed there was a reason.
Of course, Actions can do that, too.
As a side note, GitHub actions can be used to compile and test NVDA code using relatively new windows systems. It would also be possible to link crowdin to the GitHub code repository. Finally, it might be possible to automate the task of posting alpha, stable, and test boards to the website. To reduce the workload of the staff!
@XLTechie
I often wondered why NV Access didn't, especially for the slow-to-compile C components, and clonable items that hadn't been updated, but assumed there was a reason.
We've already done this in a different way for some components like https://github.com/nvaccess/nvda-cldr
The issue of AppVeyor making their instances cheaper and slower over time is still going to be an issue even if we optimize other parts of the build system. The slowest parts of the build are quite challenging to split up. We can always speed up the build system in many ways, and GitHub actions is probably the most effective solution.
@dpy013 You are right about the relatively new Windows systems. It might even allow for broader testing setups, if we can cause artifacts to be tested on several versions of Windows at once. That alone would probably make it worth doing.
Certainly, if Crowdin can be linked to AppVeyor compiles as it is now, it can be equally well linked to Actions compilations, probably with the same scripts.
Posting artifacts to the website is already automated, at least for alphas and betas.
I've started to study modules of GitHub Actions with the Study guide for a possible preparation to get the GitHub Actions certificate from GitHub. I don't know if finally I'll try to get the certifications since I need to show Government documentation, installing a software etc. But if someone doesn't take this, I may create a PRafter testing. We may use the new attestations feature too.
@nvdaes - there's no need to worry about build signing, we're currently working on changing this actively
I happened to have nothing to do in the morning, so I went and converted a version of GitHub actions using gpt4 + NVDA's appveyor.yml. Now stuck at appveyor\scripts\setBuildVersionVars.ps1 I'm stuck on this script, plus I don't know much about PowerShell Core, so anyone interested can take it and look into it. https://github.com/dpy013/nvda/blob/1c8e80ff50c4371dae40908ddcfcd45b9f0283c5/.github/workflows/NVDABuild.yaml
Currently, the one-hour time limit on AppVeyor is insufficient for compiling NVDA alpha-35333. Does the official team have any plans to implement issue 10516? If this issue is not needed, I'll prepare to convert this problem into a discussion.
@dpy013 this issue has been triaged - i.e. accepted. We plan to eventually move to GitHub actions. Community support to get this done is welcome. Please do not close or convert this issue. Please also note that alpha builds fail sometimes, and the recent failure is not related to build time.
I think that I would work on this. I used GitHub Actions previously to test add-ons with NVDA and I didn't feel comfortable to use Appveyor. Then we can discuss problems like build signing, and where to store artifacts. Recently I studied GitHub foundations module, in addition to GitHub Actions studied previously, though I didn't take GitHub certifications, and this may be useful to practice.
Perhaps this issue should be addressed after migrating to a more recent version of Python. Seems that Python 3.11 is not available for windows-latest, and trying to force an installation of Python 3.11 using a precompilled executable or from Python source code may not be clean. I tried to install Python 3.11 x86 with the setup Python action, and the workflow run has failed:
https://github.com/nvdaes/nvda/actions/runs/13439340081<
Finally I have been able to use setup Python action to install Python 3.11 x64, using windows-2025 runner on GitHub. I'll try to use windows-latest again in case. I will open a PR, though I'll require assistance to know which tags for system tests are required, at least.
I've been testing GitHub actions using powershell scripts used for appveyor, but seems that this doesn't work. Also, I've tested using the .bat files, like done in local computers, and this works. Some Chrome system tests are always failing, and I've seen some staff in powershell so that appveyor considers this fact. After testing, my approach for a preliminary PR would be to use the GitHub script action instead of Powershell to set the appropriate version, if it's a release etc., depppending on the GitHub context (sha, branch name, etc). And I'll use the .bat files, not powershell. For things like webhooks, signing or upload symbols to Mozilla, I will stop since this needs help from NV Access. I comment this in case you prefer another approach. Appveyor is used years ago and I had thought about making a migration with equivalent commands, but seems that Powershell scripts are difficult to use.
Finally I'm using powershell scripts when possible. A PR is opened (work in progress for now).