Kara Guo

A follow up (status update): Even without the additional changes mentioned at `Steps to reproduce the problem`, this issue occurred again. This means that this happens at a low possibility,...

Update: To throw the error at following place is supposed to fix the issue. https://github.com/goharbor/harbor-operator/blob/544d6737c197a5fefb9c729ae8785614f77ab005/pkg/cluster/controllers/database/readiness.go#L58

@MinerYang The source harbor is v2.6.2, and destination harbor is 2.8.4

The response of `curl -skL https://admin:@/api/v2.0/projects//repositories//artifacts/?with_accessory=true&with_signature=true` ``` root@:~# curl -skL https://admin:@/api/v2.0/projects//repositories//artifacts/?with_accessory=true&with_signature=true [1] 1392264 root@:~# {"accessories":null,"addition_links":{"build_history":{"absolute":false,"href":"/api/v2.0/projects//repositories//artifacts/sha256:/additions/build_history"}},"annotations":{},"digest":"sha256:","extra_attrs":{"architecture":"amd64","author":"","config":{"Cmd":["nginx","-g","daemon off;"],"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Labels":{},"StopSignal":"SIGQUIT","User":"nginx","Volumes":{"/run":{},"/var/cache/nginx":{},"/var/log/nginx":{}},"WorkingDir":"/"},"created":"2023-12-08T19:51:53.73133776Z","os":"linux"},"icon":"sha256:","id":1938,"labels":null,"manifest_media_type":"application/vnd.docker.distribution.manifest.v2+json","media_type":"application/vnd.docker.container.image.v1+json","project_id":21,"pull_time":"0001-01-01T00:00:00.000Z","push_time":"2024-01-08T03:03:45.789Z","references":null,"repository_id":71,"size":49740265,"tags":[{"artifact_id":1938,"id":1931,"immutable":false,"name":"v2.8.4-gke.4","pull_time":"0001-01-01T00:00:00.000Z","push_time":"2024-01-08T03:03:45.932Z","repository_id":71,"signed":false}],"type":"IMAGE"} ^C [1]+ Done curl -skL https://admin:[email protected]:10443/api/v2.0/projects/gpc-system-container-images/repositories/private-cloud-staging%2Fgoharbor%2Fharbor-portal/artifacts/v2.8.4-gke.4?with_accessory=true ``` And the replication rule looks like ![Screenshot...

Sure in the storage, we can see the sig manifest ``` harbor [ /var/lib/registry/docker/registry/v2/blobs/sha256/67 ]$ cat /data {"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","size":233,"digest":"sha256:"},"layers":[{"mediaType":"application/vnd.dev.cosign.simplesigning.v1+json","size":304,"digest":"sha256:","annotations":{"dev.cosignproject.cosign/signature":"=","dev.sigstore.cosign/certificate":"-----BEGIN CERTIFICATE----------END CERTIFICATE-----\n","dev.sigstore.cosign/chain":""}}],"annotations":{, ".tag":"sha256-32b16e80.sig"}} ``` From dest harbor, the artifact show no sig...

I found a log that confirms my assumption that the sig manifest is pushed before its artifact manifest, and when the artifact is pushed, the artifact doesn't include the sig...

@MinerYang the replication task log: ``` 2024-01-10T22:29:58Z [INFO] [/controller/replication/transfer/image/transfer.go:139]: client for source registry [type: harbor, URL: http://core:8080, insecure: true] created 2024-01-10T22:29:58Z [INFO] [/controller/replication/transfer/image/transfer.go:149]: client for destination registry [type: harbor, URL:...

@Vad1mo Thanks for the comments! Sgtm! I can work on a brief demo recording and follow up with the review process to kick off

It is because all containers including `registry` and `registryctl` in deployment `registry` are populated with the resource configuration values provided by `registry` component spec, as [code](https://github.com/karaguo/harbor-operator/blob/12cb7d7034873e127626351fdd0170e0a059554a/controllers/goharbor/registry/deployments.go#L345) and [code](https://github.com/karaguo/harbor-operator/blob/544d2dcbee6cd3e77236df387ba6f3de4f76d784/apis/meta/v1alpha1/component.go#L116). The configuration...

It looks that the https://github.com/distribution/distribution/issues/4478 is quite silimiar, and the fix is in recently