bytecodealliance
[RFC] Predictable and (almost) fully automated WAMR releases
Feature
I'd like to discuss with the community standardizing release cadence of WAMR releases.
Benefit
The customers can know in advance any WAMR releases and plan accordingly. Customers no longer have to request ad-hoc requests.
Implementation
As a first step we need to make sure all of the tests are automated and publicly available, so Intel engineers no longer have to be overloaded with release requests; any maintainer can coordinate the release. Based on the discussion we had in the community meetings, there are two major actions to achieve this:
- decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)
- enable tests that require QEMU
Whereas it's unknown yet whether 2. can be implemented completely using github actions, I think the problem overall is solvable one way or another (e.g. by using instances from 3rd party cloud providers etc.) so whoever picks this up can figure out the implementation detail. The 1. can look a bit concerning, so I'd like to clarify that I'm not suggesting for it to completely decouple IDE from the WAMR project but instead keep it around with the same versioning schema, but make releases on best-effort basis.
The regular releases will include any changes that happened since the previous release, and if the particular feature will not be ready, it'll have to wait for another release (hence I'm proposing relatively frequent releases).
When it comes to the versioning, I propose incrementing MINOR version by default and change it when:
- there are mainly bugfixes and minor improvements, only the PATCH number should be increased
- for any breaking changes, MAJOR version should be incremented
I guess the preference would be not to change the MAJOR version too often, so for that we should likely maintain a dev branch where eventually all the breaking changes are included, and then added to the main branch through a single pull request to make sure they all are included in a single release.
This is just a proposal and I do understand it might not touch on all the potential problems and details. I'm open for feedback and suggestions; feel free to leave your comment below.
- decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)
Agreed. As a tool used during the development phase, the cost of rolling out IDE updates to end-users is significantly lower compared to the core runtime, allowing for a faster and more flexible release cadence.
enable tests that require QEMU
If possible, we should incorporate as many automated tests as we can. Ideally, if quality can be guaranteed, we should be able to perform a release operation at any commit point. Of course, it is also necessary to take into account the features and bug fixes included in certain versions.
decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)
Even inclined to add "lldb wamr patched" to a list of items that are only released if changes have been made.
When it comes to the versioning, I propose incrementing MINOR version by default and change it when: there are mainly bugfixes and minor improvements, only the PATCH number should be increased for any breaking changes, MAJOR version should be incremented I guess the preference would be not to change the MAJOR version too often, so for that we should likely maintain a dev branch where eventually all the breaking changes are included, and then added to the main branch through a single pull request to make sure they all are included in a single release.
Agreed. This approach makes it possible for a GitHub action to determine the next version string.
It is just a suggested list for the CI improvement plan, not the final version. It is intended for brainstorming and discussion. The goal is to enhance the current manual release process, improve PR review efficiency, and strengthen quality control.
- [ ] As the starting point, automatically create a PR to update version information in both version.cmake and version.h, and generate new content in RELEASE.md. #4502
- [ ] Once the PR is approved, the entire release process begins.
- [ ] The entire release process should also be able to be triggered manually. This is useful when a previous release attempt has failed. In this mode, it is necessary to remove the tag and the release created by the failed attempt.
All items below will be tracked elsewhere. Focus on automation in this PR.
- [ ] At the very least, maintain the same compilation flag groups used by release binaries (iwasm, wamrc) in the Windows CI.
- [ ] At the very least, maintain the same compilation flag groups used by release binaries (iwasm, wamrc) in the MacOS CI.
- [ ] ❓ Use pre-compiled LLVM library binaries released by the llvm-project to replace the LLVM libraries used by iwasm and wasm, in order to save CI time consumption.
- [ ] Refactor compilation option groups for iwasm. The goal is to improve (features, flag)coverage and try to make it more efficient.
- [ ]
WASM_CPU_SUPPORTS_UNALIGNED_ADDR_ACCESS - [ ] 32-bit
- [ ] TBD
- [ ]
- [ ] Add more containers for embedding testing environments.
- [ ] Those are from Embedded-SIG-Target-Platforms
- [ ] Xtensa
- [ ] RISC-V 32
- [ ] ARM 32
- [ ] RISC-V 64
- [ ] ARM 64
- [ ] We are aware of
- [ ] ARC
- [ ] Also are aware of
- [ ] Zephyr
- [ ] ESP-IDF
- [ ] Those are from Embedded-SIG-Target-Platforms
- [ ] May not release wamr-sdk every time
- [ ] Reduce packed binaries for each platform. You may keep .tar.gz for Ubuntu and macOS, and .zip for Windows.
- [ ] Improve CI on popular development environments, which may include Windows MSVC, Windows MinGW, macOS Intel, macOS ARM, and Ubuntu.