karmada icon indicating copy to clipboard operation
karmada copied to clipboard

image-scanning: fix ratelimiting error when downloading vulnerability db from ghcr.io

Open zhzhuang-zju opened this issue 1 year ago • 6 comments

What type of PR is this? /kind cleanup

What this PR does / why we need it: Recently, the image-scanning CI encountered a rate-limiting error when downloading the vulnerability database from ghcr.io. See:

  • https://github.com/karmada-io/karmada/actions/runs/11270619338

There is an issue https://github.com/aquasecurity/trivy-action/issues/389 tracking this problem in the trivy-action repository. Released v0.26.0 which adds support for caching should alleviate some of the pain as caching should ensure DBs are reused if cache is available.

In addition, the following have also been strengthened:

  • specify multiple DB registries, which will try the default GitHub Registry, and if too many requests is reached, will use the aws mirror
  • avoid repeatedly updating the Vulnerability DB

Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


zhzhuang-zju avatar Oct 11 '24 07:10 zhzhuang-zju

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 43.16%. Comparing base (6f138cf) to head (d2c29f0).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5673   +/-   ##
=======================================
  Coverage   43.15%   43.16%           
=======================================
  Files         658      658           
  Lines       56006    56006           
=======================================
+ Hits        24170    24173    +3     
+ Misses      30265    30263    -2     
+ Partials     1571     1570    -1     
Flag Coverage Δ
unittests 43.16% <ø> (+<0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Oct 11 '24 08:10 codecov-commenter

cc @liangyuanpeng

zhzhuang-zju avatar Oct 11 '24 08:10 zhzhuang-zju

Is this the official solution? I see the issue(https://github.com/aquasecurity/trivy-action/issues/389) is still open there.

RainbowMango avatar Oct 11 '24 08:10 RainbowMango

Is this the official solution? I see the issue(aquasecurity/trivy-action#389) is still open there.

I'm not sure why this issue is still open. However, upgrading to v0.26.0 does help mitigate the problem, as it offers caching capabilities that can effectively reduce the number of requests. Of course, we can also continue to wait for official progress.

zhzhuang-zju avatar Oct 11 '24 08:10 zhzhuang-zju

local verification: https://github.com/zhzhuang-zju/karmada/actions/runs/11288078707/job/31395167983 Caching mechanism: https://github.com/aquasecurity/trivy-action?tab=readme-ov-file#cache

I've noticed that trivy-action has just released version v0.27.0. We can wait until the official version is stable before proceeding further.

zhzhuang-zju avatar Oct 11 '24 08:10 zhzhuang-zju

Just record the failing tests here:

  • 2024.10.25: https://github.com/karmada-io/karmada/actions/runs/11512908408/job/32048690637
  • 2024.10.28: https://github.com/karmada-io/karmada/actions/runs/11547003482/job/32136213459
  • 2024.10.29: https://github.com/karmada-io/karmada/actions/runs/11565828027/job/32193466183
  • 2024.10.31: https://github.com/karmada-io/karmada/actions/runs/11608402982/job/32323586211
  • 2024.11.02: https://github.com/karmada-io/karmada/actions/runs/11642256091/job/32421651069
  • 2024.11.04: https://github.com/karmada-io/karmada/actions/runs/11656444953/job/32452565021
  • 2024.11.05: https://github.com/karmada-io/karmada/actions/runs/11675955462/job/32511394986
  • 2024.11.08: https://github.com/karmada-io/karmada/actions/runs/11739946163/job/32705497376
  • 2024.11.12: https://github.com/karmada-io/karmada/actions/runs/11739946163/job/32705497376
  • 2024.11.12: https://github.com/karmada-io/karmada/actions/runs/11790423342/job/32840827117
  • 2024.11.14: https://github.com/karmada-io/karmada/actions/runs/11829163113/job/32960501449
  • 2024.11.14: https://github.com/karmada-io/karmada/actions/runs/11829163113/job/32960501449

RainbowMango avatar Oct 25 '24 10:10 RainbowMango

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

karmada-bot avatar Nov 14 '24 06:11 karmada-bot