cncf
Requesting use of cncf/cluster resources for Hyperledger Fabric CI test and validation
Hello, CNCF/cluster!
Ry Jones from the Hyperledger Foundation steered us towards the cncf/cluster as a possible foundation for running CI, build, validation, and performance evaluations for the Hyperledger Fabric platform. The project is 100% open-source, managed under the umbrella of sponsorship by the Linux Foundation, and is a "real project" advanced as a community effort to deliver enterprise-grade permissioned blockchain systems.
During 2022, Fabric shifted the bulk of the CI, test, and source code approval workflows from Azure to GitHub. This has been a great step forward in alignment, consumability, and community access to the platform, but has a few negative side-effects still remain:
-
Integration, system tests, and E2E sample validation pipelines commonly take hours to run. (The bulk of the validation workflow phases are spent waiting for GitHubActions executors in a queued state.) This is disruptive for engineers working on pull requests, which typically need some level of interactive re-runs on failed PRs, code reviews, and re-submission of test jobs.
-
GitHub executors are constrained to 7GRAM systems. One would imagine that 7 Gigabytes (!!) of RAM on a system would be enough for validation of test workflows, but with this limit in place, several of our Kubernetes-native integration scenarios do not have sufficient resources to run successfully. For k8s integration testing, we have had very good luck running with ephemeral KIND clusters, deploying k8s operators, k8s resources, networks, testing, and tearing everything down at the end of the test runs.
If the cncf/cluster is amenable, we would like to try supplementing the GitHub Actions build executors with systems running at the shared cluster. Resources would be triggered by CI activities (developer checks in code, submits a PR, merges a branch, turns a release, etc.) and bound to commits performed at the github.com/hyperledger organization. Workloads would be container-based, run in a sandbox, require compute resources only (no long-term or network-node storage), and scale according to the commit pace on the overall repo.
