harbor icon indicating copy to clipboard operation
harbor copied to clipboard
verified publisher icon goharbor

Event-based P2P Preheat execution does not happen for pull-through proxy projects

Open valkolaci opened this issue 1 year ago • 5 comments

If you are reporting a problem, please make sure the following information are provided:

Expected behavior and actual behavior: Event-based P2P Preheat execution does not happen for pull-through proxy projects

Steps to reproduce the problem:

[root@testenv ~]# mkdir harbor
[root@testenv ~]# mkdir /data
[root@testenv ~]# cd harbor
[root@testenv harbor]# wget https://github.com/goharbor/harbor/releases/download
[root@testenv harbor]# tar xvzf harbor-offline-installer-v2.12.2.tgz
[root@testenv harbor]# cd harbor
[root@testenv harbor]# cp harbor.yml.tmpl harbor.yml

## edit harbor.yml:

 hostname: testenv.example.com
 http:
   port: 8080
# https:
#  port: 443
#  certificate: /your/certificate/path
#  private_key: /your/private/key/path

log:
  level: debug

[root@testenv harbor]# ./prepare
prepare base dir is set to /root/harbor/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http prot
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@testenv harbor]# ./install.sh --with-trivy

[Step 0]: checking if docker is installed ...

Note: docker version: 28.0.4

[Step 1]: checking docker-compose is installed ...

Note: Docker Compose version v2.34.0

[Step 2]: loading Harbor images ...
Loaded image: goharbor/redis-photon:v2.12.2
Loaded image: goharbor/nginx-photon:v2.12.2
Loaded image: goharbor/registry-photon:v2.12.2
Loaded image: goharbor/prepare:v2.12.2
Loaded image: goharbor/harbor-portal:v2.12.2
Loaded image: goharbor/harbor-core:v2.12.2
Loaded image: goharbor/harbor-jobservice:v2.12.2
Loaded image: goharbor/harbor-registryctl:v2.12.2
Loaded image: goharbor/harbor-log:v2.12.2
Loaded image: goharbor/harbor-db:v2.12.2
Loaded image: goharbor/harbor-exporter:v2.12.2
Loaded image: goharbor/trivy-adapter-photon:v2.12.2


[Step 3]: preparing environment ...

[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http prot
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf   
Clearing the configuration file: /config/registry/passwd 
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env 
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env  
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/portal/nginx.conf  
Generated configuration file: /config/log/logrotate.conf 
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf   
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /config/trivy-adapter/env  
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir


Note: stopping existing Harbor instance ...

[Step 5]: starting Harbor ...
[+] Running 11/11
 ✔ Network harbor_harbor        Created
 ✔ Container harbor-log         Started
 ✔ Container harbor-db          Started
 ✔ Container registry           Started
 ✔ Container registryctl        Started
 ✔ Container harbor-portal      Started
 ✔ Container redis              Started
 ✔ Container trivy-adapter      Started
 ✔ Container harbor-core        Started
 ✔ Container harbor-jobservice  Started
 ✔ Container nginx              Started
✔ ----Harbor has been installed and started successfully.----
  1. Log in to UI
  2. Administration / Registries: New Endpoint:
  • Provider: Github GHCR
  • Name: github
  1. Administration / Distributions: New Instance:
  • Provider: Dragonfly
  • Name: dragonfly
  • Endpoint: ...
  1. Projects: New Project:
  • Project Name: github
  • Access Level: Public
  • Proxy Cache: yes, github
  1. Projects: github, set Configuration:
  • Deployment security: Prevent vulnerable images of Critical and above
  • Vulnerability scanning: Automatically scan images on push
  1. Projects: github, set P2P Preheat: New policy:
  • Provider: dragonfly
  • Name: preheat
  • Filters: Repositories: **
  • Filters: Tags: **
  • Criteria: No vulnerability severity of Critical and above
  • Trigger: Event based
  • Scope: Single peer
  1. Projects: library, set Configuration:
  • Deployment security: Prevent vulnerable images of Critical and above
  • Vulnerability scanning: Automatically scan images on push
  1. Projects: library, set P2P Preheat: New policy:
  • Provider: dragonfly
  • Name: preheat
  • Filters: Repositories: **
  • Filters: Tags: **
  • Criteria: No vulnerability severity of Critical and above
  • Trigger: Event based
  • Scope: Single peer
[root@testenv harbor]# docker login testenv.example.com:8080 -u admin -p Harbor12345
...
[root@testenv harbor]# docker pull testenv.example.com:8080/github/linuxserver/nginx:latest
...
[root@testenv harbor]# docker tag testenv.example.com:8080/github/linuxserver/nginx:latest testenv.exam
...
[root@testenv harbor]# docker push testenv.example.com:8080/library/nginx:latest
...

Both Project library and Project github has the same Vulnerability scanning and P2P Preheat configuration. Both Projects will execute vulnerability scanning. But only Project library will send the scanned image to P2P Preheating after a successful scan.

Versions: Please specify the versions of following systems.

  • harbor version: v2.12.2
  • docker engine version: 28.0.4
  • docker-compose version: v2.34.0

valkolaci avatar Apr 04 '25 10:04 valkolaci

Some relevant logs:

Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/notifier/event/event.go:112]: event SCANNING_COMPLETED published
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/controller/event/handler/p2p/preheat.go:95]: preheat: image scanned github/linuxserver/nginx:
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/config/manager.go:142]: failed to get key audit_log_forward_endpoint, error: the configure value is not set, maybe default value not defined before get
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/controller/event/handler/webhook/scan/scan.go:62]: Cannot find policy for SCANNING_COMPLETED event: Artifact-&{NamespaceID:2 Repository:github/linuxserver/nginx Tag: Digest:sha256:7b2627afa627c087201c297d2270d5bac95d06e8fa6f3408526d457f2a5b07dc MimeType:application/vnd.oci.image.manifest.v1+json Size:0} Operator-harbor#proxy-cache-service OccurAt-2025-04-04 09:37:29
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/notifier/notifier.go:220]: Handle notification with Handler 'ScanWebhook' on topic 'SCANNING_COMPLETED': Artifact-&{NamespaceID:2 Repository:github/linuxserver/nginx Tag: Digest:sha256:7b2627afa627c087201c297d2270d5bac95d06e8fa6f3408526d457f2a5b07dc MimeType:application/vnd.oci.image.manifest.v1+json Size:0} Operator-harbor#proxy-cache-service OccurAt-2025-04-04 09:37:29
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/allowlist/manager.go:75]: No CVE allowlist found for project 0, returning empty list.
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [INFO] [/controller/p2p/preheat/enforcer.go:281]: Policy github.preheat's criteria 'vulnerability' uses value 'Critical:5' from project configurations
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[0]: type=repository, value=**
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[1]: type=tag, value=**
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[2]: type=vulnerability, value=5
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:70]: Preheat filter[0] input: [0] candidates
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:77]: Preheat filter[0] output: [0] candidates
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/controller/p2p/preheat/enforcer.go:301]: No preheat policy matched for the artifact github/linuxserver/nginx@sha256:7b2627afa627c087201c297d2270d5bac95d06e8fa6f3408526d457f2a5b07dc
Apr  4 11:37:29 172.18.0.1 core[103569]: 2025-04-04T09:37:29Z [DEBUG] [/pkg/notifier/notifier.go:220]: Handle notification with Handler 'P2PPreheat' on topic 'SCANNING_COMPLETED': Artifact-&{NamespaceID:2 Repository:github/linuxserver/nginx Tag: Digest:sha256:7b2627afa627c087201c297d2270d5bac95d06e8fa6f3408526d457f2a5b07dc MimeType:application/vnd.oci.image.manifest.v1+json Size:0} Operator-harbor#proxy-cache-service OccurAt-2025-04-04 09:37:29

valkolaci avatar Apr 04 '25 10:04 valkolaci 

Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/notifier/event/event.go:112]: event SCANNING_COMPLETED published
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/controller/event/handler/p2p/preheat.go:95]: preheat: image scanned library/nginx:latest
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/config/manager.go:142]: failed to get key audit_log_forward_endpoint, error: the configure value is not set, maybe default value not defined before get
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/controller/event/handler/webhook/scan/scan.go:62]: Cannot find policy for SCANNING_COMPLETED event: Artifact-&{NamespaceID:1 Repository:library/nginx Tag:latest Digest:sha256:44604568e1c33d6ed49c659de7391020ea4cac85484517a5f2ba466b86cedf92 MimeType:application/vnd.docker.distribution.manifest.v2+json Size:0} Operator-admin OccurAt-2025-04-04 09:40:20
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/notifier/notifier.go:220]: Handle notification with Handler 'ScanWebhook' on topic 'SCANNING_COMPLETED': Artifact-&{NamespaceID:1 Repository:library/nginx Tag:latest Digest:sha256:44604568e1c33d6ed49c659de7391020ea4cac85484517a5f2ba466b86cedf92 MimeType:application/vnd.docker.distribution.manifest.v2+json Size:0} Operator-admin OccurAt-2025-04-04 09:40:20
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/notifier/event/event.go:112]: event DELETE_ROBOT published
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/allowlist/manager.go:75]: No CVE allowlist found for project 0, returning empty list.
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [INFO] [/controller/p2p/preheat/enforcer.go:281]: Policy library.dragonfly's criteria 'vulnerability' uses value 'Critical:5' from project configurations
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[0]: type=repository, value=**
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[1]: type=tag, value=**
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:107]: Build preheat filter[2]: type=vulnerability, value=5
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:70]: Preheat filter[0] input: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:77]: Preheat filter[0] output: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:70]: Preheat filter[1] input: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:77]: Preheat filter[1] output: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:70]: Preheat filter[2] input: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/policy/filter.go:77]: Preheat filter[2] output: [1] candidates
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/notifier/notifier.go:220]: Handle notification with Handler 'AuditLog' on topic 'DELETE_ROBOT': Name-robot$library+TYxQ26wZ-Trivy-d086c487-1138-11f0-af06-f6b2c0dbe68c Operator-harbor-jobservice OccurAt-2025-04-04 09:40:20
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/p2p/preheat/provider/client/http_client.go:91]: Get http://mgmt1:8080/healthy with params=map[], options=map[] succeed: "OK"
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/task/task.go:110]: the database record for task 21 created
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/task/task.go:127]: the task 21 is submitted to jobservice, the job ID is 24921d23c0c8682936b4b041
Apr  4 11:40:20 172.18.0.1 core[103569]: 2025-04-04T09:40:20Z [DEBUG] [/pkg/notifier/notifier.go:220]: Handle notification with Handler 'P2PPreheat' on topic 'SCANNING_COMPLETED': Artifact-&{NamespaceID:1 Repository:library/nginx Tag:latest Digest:sha256:44604568e1c33d6ed49c659de7391020ea4cac85484517a5f2ba466b86cedf92 MimeType:application/vnd.docker.distribution.manifest.v2+json Size:0} Operator-admin OccurAt-2025-04-04 09:40:20

valkolaci avatar Apr 04 '25 10:04 valkolaci

I'll try to reproduce the issue in my environment, based on the information you provided, the preliminary diagnosis is that P2P event triggering is normal under normal projects, but abnormal for proxy cache type projects.

chlins avatar Apr 07 '25 06:04 chlins

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] avatar Jun 06 '25 09:06 github-actions[bot]

up

valkolaci avatar Jun 10 '25 02:06 valkolaci

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] avatar Aug 10 '25 09:08 github-actions[bot]

up

valkolaci avatar Aug 22 '25 14:08 valkolaci

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] avatar Oct 22 '25 09:10 github-actions[bot]