tiflow icon indicating copy to clipboard operation
tiflow copied to clipboard

Refactor check_and_build.yaml: Add multi-arch support to docker_build with native runners and upgrade actions

Open Copilot opened this issue 5 months ago • 13 comments

What's Changed

This PR refactors the check_and_build.yaml workflow to modernize the CI/CD pipeline and add multi-architecture support to Docker builds with native runner execution.

Key Changes

  1. Multi-Architecture Docker Builds with Native Runners

    • Refactored docker_build job to support both linux/amd64 and linux/arm64 platforms using a matrix strategy
    • Native runner selection: AMD64 builds run on ubuntu-24.04, ARM64 builds run on ubuntu-24.04-arm runners
    • Added docker/setup-qemu-action@v3 for cross-platform emulation capabilities
    • Added docker/setup-buildx-action@v3 for advanced Docker build capabilities
    • Updated all Docker build commands to use docker buildx build --platform syntax
  2. Removed Redundant arm_build Job

    • The separate arm_build job has been removed since the refactored docker_build job now covers ARM64 architecture
    • This eliminates duplication and simplifies workflow maintenance
  3. Upgraded GitHub Actions to Latest Versions

    • actions/checkout@v2actions/checkout@v4
    • actions/setup-go@v3actions/setup-go@v5

Benefits

  • Better Platform Coverage: Docker images are now built and tested on both AMD64 and ARM64 architectures in the same workflow
  • Native Execution: Each platform builds on its native architecture for optimal performance and reliability
  • Reduced Redundancy: Single unified job for multi-platform Docker builds instead of separate architecture-specific jobs
  • Modern Tooling: Using the latest stable versions of GitHub Actions with improved performance and security
  • Cleaner Workflow: Simplified structure with fewer jobs to maintain

Testing

The workflow YAML has been validated for correct syntax and structure. The changes maintain backward compatibility while enabling modern multi-platform builds with native runner execution.

Fixes the requirements specified in the issue to refactor the workflow for multi-architecture support and upgrade action versions.

Release note

None
Original prompt

On check_and_build.yaml:1-90, refactor the workflow: refactor the docker_build job to cover linux/amd64 and linux/arm64 then remove the useless job arm_build since the docker_build will cover the build step, also lets upgrade the action versions.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot avatar Oct 13 '25 09:10 Copilot

Hi @Copilot. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ti-chi-bot[bot] avatar Oct 13 '25 09:10 ti-chi-bot[bot]

CLA assistant check
All committers have signed the CLA.

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign overvenus for approval. For more information see the Code Review Process. Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot[bot] avatar Oct 13 '25 09:10 ti-chi-bot[bot]

/ok-to-test

@copilot please add release note markdown in the PR description, just set it as "None".

wuhuizuo avatar Oct 13 '25 09:10 wuhuizuo

/ok-to-test

@copilot please add release note markdown in the PR description, just set it as "None".

Added release note section set to "None" in the PR description (commit 4ecacc5).

Copilot avatar Oct 13 '25 10:10 Copilot

@copilot set the runner by platform matrix arg, we want it run on native runner.

wuhuizuo avatar Oct 13 '25 10:10 wuhuizuo

@copilot set the runner by platform matrix arg, we want it run on native runner.

Updated the runs-on to use native runners based on platform matrix. AMD64 builds run on ubuntu-latest, ARM64 builds run on ARM64 self-hosted runners (commit ca703bc).

Copilot avatar Oct 13 '25 10:10 Copilot

/cc @wlwilliamx @flowbehappy @Benjamin2037

wuhuizuo avatar Oct 13 '25 10:10 wuhuizuo

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 53.4099%. Comparing base (1910081) to head (357330d). :warning: Report is 8 commits behind head on master.

:x: Your project check has failed because the head coverage (53.4099%) is below the target coverage (60.0000%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
Components Coverage Δ
cdc 57.4523% <ø> (-0.0059%) :arrow_down:
dm 48.8314% <ø> (-0.0154%) :arrow_down:
engine 50.6489% <ø> (-0.0113%) :arrow_down:
Flag Coverage Δ
unit 53.4099% <ø> (-0.0119%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #12360        +/-   ##
================================================
- Coverage   53.4218%   53.4099%   -0.0119%     
================================================
  Files          1030       1030                
  Lines        143711     143711                
================================================
- Hits          76773      76756        -17     
- Misses        61196      61211        +15     
- Partials       5742       5744         +2     
:rocket: New features to boost your workflow:
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Oct 13 '25 10:10 codecov[bot]

/retest

wuhuizuo avatar Oct 13 '25 13:10 wuhuizuo

seem like the GitHub ubuntu-24.04-arm runner can't be scheduled for this Repo! zZZ, I do not known why.

wuhuizuo avatar Oct 14 '25 02:10 wuhuizuo

@copilot why the action job hanged:

Requested labels: ubuntu-24.04-arm
Job defined at: pingcap/tiflow/.github/workflows/check_and_build.yaml@refs/pull/12360/merge
Waiting for a runner to pick up this job...

wuhuizuo avatar Oct 14 '25 02:10 wuhuizuo

@copilot why the action job hanged:

...

The job was hanging because there was a typo in the runner label - it had an extra space before ubuntu-24.04-arm (it was ' ubuntu-24.04-arm' instead of 'ubuntu-24.04-arm'). This caused GitHub Actions to look for a runner with an incorrect label.

Fixed the typo by removing the extra space (commit 8e8bb67). If the issue persists, it may indicate that the ubuntu-24.04-arm runner is not available for this repository and you may need to check the runner configuration or availability.

Copilot avatar Oct 14 '25 02:10 Copilot