intra-step leasing: Add lease proxy server specs in prowgen
This is the preparatory work that ci-operator requires in order to run a small lease proxy server.
We are working on the intra-step leasing capability that will allow a step from the ci-operator step registry to acquire lease from boskos.
As of today, ci-operator acquire any leases required by a workflow and then run the test. With the new feature we are about to introduce, a step could also do that by sending requests to ci-operator that will, in turn, forward them to boskos.
A typical scenario looks like so:
-
ci-operatoris supposed to run the e2e-aws-ovn test:
- as: e2e-aws-ovn
skip_if_only_changed: (^(docs|upi|hack)/)|((^|/)OWNERS(_ALIASES)?$)|((^|/)[A-Z]+\.md$)|(^images/(baremetal|libvirt|openstack))|(^cluster-api/providers/([^a]|a[^w]|aw[^s]).*/)|(/agent/)
steps:
cluster_profile: aws-4
env:
AWS_INSTALL_USE_MINIMAL_PERMISSIONS: "yes"
USER_TAGS: |
keyA valueA
key2 value2
keySpecial v@lu3-Special_.:/=+
keyD valueD spaces
workflow: openshift-e2e-aws
- The test uses a
cluster_profile, thereforeci-operatorruns a small HTTP server that acts as a lease proxy server, actually forwarding any requests toboskos. - The workflow
openshift-e2e-awsexecutes. - Every step of the workflow has the
CI_OPERATOR_POD_IPenvironment variable set, that it might uses to send requests to the proxy run byci-operator. - The
ipi-install-installstep runs and tries to acquire a lease from the proxy. -
ci-operatorreceives the request, forwards it toboskosand finally returns the result back to theipi-install-installstep.
ipi-install-install.sh step
|
HTTP POST `/acquire?type=aws-1-qe-quota-slice&count=1`
|
+----> lease-proxy-server run by `ci-operator`
|
HTTP POST `/acquire?type=aws-1-qe-quota-slice&count=1`
|
+----> `boskos`
Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.
For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.
This repository is configured in: automatic mode
Walkthrough
Adds a lease-proxy port and pod IP env: a new exported constant (8082), a PodSpec mutator that injects a lease-proxy port and CI_OPERATOR_POD_IP env, updates prowgen test builder to apply it, and updates related test fixtures and generated job YAMLs.
Changes
| Cohort / File(s) | Summary |
|---|---|
Constants pkg/api/constant.go |
Added exported constant LeaseProxyServerPort = 8082. |
Pod spec mutator pkg/prowgen/podspec.go |
Added helper addPort(), package var leaseProxyServerEnv, and exported LeaseProxyServer() PodSpecMutator to add a lease-proxy port and inject CI_OPERATOR_POD_IP. |
Prow job builder pkg/prowgen/jobbase.go |
NewProwJobBaseBuilderForTest now applies LeaseProxyServer() when a cluster profile is present for multi-stage test configs (in addition to existing LeaseClient). |
prowgen test fixtures pkg/prowgen/testdata/zz_fixture_...yaml |
Added CI_OPERATOR_POD_IP env (from fieldRef.status.podIP) and container port 8082 named lease-proxy. |
ephemeralcluster controller fixtures pkg/controller/ephemeralcluster/testdata/zz_fixture_pj_...yaml |
Added CI_OPERATOR_POD_IP env and lease-proxy port (8082) to ProwJob podSpecs. |
Integration / generated job YAMLs test/integration/**, test/integration/pj-rehearse/**, test/integration/repo-init/** |
Updated multiple generated and expected presubmit job YAMLs to include CI_OPERATOR_POD_IP env and a lease-proxy container port 8082 across affected jobs. |
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
✨ Finishing touches
- [ ] 📝 Generate docstrings
[!WARNING] Tools execution failed with the following error:
Failed to run tools: 13 INTERNAL: Received RST_STREAM with code 2 (Internal server error)
Comment @coderabbitai help to get the list of available commands and usage tips.
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: danilo-gemoli
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [danilo-gemoli]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
/label tide/merge-method-squash
Scheduling required tests: /test e2e
Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test integration-optional-test
@danilo-gemoli: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| ci/prow/breaking-changes | 63e843182aacdd08eb51e2f05b99471d22a7d5b8 | link | false | /test breaking-changes |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.