kpt
kpt copied to clipboard
Published
20 hours ago •
kptdev
kptdev
porch: packagevariant creating extra revisions
Expected behavior
PackageVariant will create a single downstream.
Actual behavior
Usually it works. But I have this weirdness:
Repositories
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ k get repositories
NAME TYPE CONTENT DEPLOYMENT READY ADDRESS
nephio-packages git Package True https://github.com/nephio-project/nephio-packages.git
nephio-testorg-packages git Package True https://github.com/johnbelamaric/nephio-testorg-packages.git
us-central1-edge-01 git Package true True https://github.com/nephio-test/jbelamaric-us-central1-edge-01.git
us-central1-edge-02 git Package true True https://github.com/nephio-test/jbelamaric-us-central1-edge-02.git
us-central1-edge-03 git Package true True https://github.com/nephio-test/jbelamaric-us-central1-edge-03.git
us-central1-edge-04 git Package true True https://github.com/nephio-test/jbelamaric-us-central1-edge-04.git
us-central1-regional git Package true True https://github.com/nephio-test/jbelamaric-us-central1-regional.git
us-central2-regional git Package true True https://github.com/nephio-test/jbelamaric-us-central2-regional.git
us-east1-edge-01 git Package true True https://github.com/nephio-test/jbelamaric-us-east1-edge-01.git
us-east1-edge-02 git Package true True https://github.com/nephio-test/jbelamaric-us-east1-edge-02.git
us-east1-regional-01 git Package true True https://github.com/nephio-test/jbelamaric-us-east1-regional-01.git
us-east1-regional-02 git Package true True https://github.com/nephio-test/jbelamaric-us-east1-regional-02.git
us-east2-regional git Package true True https://github.com/nephio-test/jbelamaric-us-east2-regional.git
PackageVariant resources (created by PackageVariantSet controller):
NAME AGE
cachingdns-distribution-1e718cfbdb754e386474da9aa91bd159ed27c6d7 25m
cachingdns-distribution-52cb15c06dc9252e966897bde315bc6b2e06af4f 25m
cachingdns-distribution-54154c218b775b84e280258f12f684c813e76c49 25m
cachingdns-distribution-aa0bd84da557680b052811b54e528625c13907cf 25m
cachingdns-distribution-d1579cce150353b437bdaa9845702b33f0900cdc 25m
cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7 25m
cachingdns-edge-3cc7f3a7d7f8d27acc6375cba42b5cc168d70fa0 25m
cachingdns-edge-44f604043046c915ea89667128b276b9c03211b7 25m
cachingdns-edge-596ce7b3b682f6eb284d7c1aec74aaaf179ad834 25m
cachingdns-edge-6b3bac48e67223051e4e47b5c52095eda9c1ee35 25m
cachingdns-edge-edf34f330bd64333cb936090fb4682e1d84f0159 25m
Those both look fine. But, here are my rpkgs:
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ kpt alpha rpkg get | grep packagevariant
us-central1-edge-01-fc057482816768663fef81c4e5252856a632751d cachingdns-edge packagevariant-1 false Draft us-central1-edge-01
us-central1-edge-02-06a45d5516402094e3150d7fd24ae5ab857a2cf2 cachingdns-distribution packagevariant-4 false Draft us-central1-edge-02
us-central1-edge-02-3df1fdf1ccfd97e68b3bf998220c3a2948b10e01 cachingdns-distribution packagevariant-6 false Draft us-central1-edge-02
us-central1-edge-02-93594975a009c248ddf9dd362d7cdd5e5ff6e0e5 cachingdns-distribution packagevariant-2 false Draft us-central1-edge-02
us-central1-edge-02-9f4cb6e8866d95d3fe3d1ab30b2ddf70621005e8 cachingdns-distribution packagevariant-5 false Draft us-central1-edge-02
us-central1-edge-02-d3b22ec4000657ed8ce88ffc6d7551ba4561c101 cachingdns-distribution packagevariant-3 false Draft us-central1-edge-02
us-central1-edge-02-f399f895cabbe912a980f5428a88b4720d9b583e cachingdns-distribution packagevariant-1 false Draft us-central1-edge-02
us-central1-edge-02-0762cce1279792c52a4d42be1368ace3e3adc136 cachingdns-edge packagevariant-33 false Draft us-central1-edge-02
us-central1-edge-02-09e084d02cdace5f4b8c91c440f5da93fcb95129 cachingdns-edge packagevariant-6 false Draft us-central1-edge-02
us-central1-edge-02-0fb62c5023354f87b6f1b7479c5ca85a960509aa cachingdns-edge packagevariant-35 false Draft us-central1-edge-02
us-central1-edge-02-1016bd461c10fd57133b7332a0296311d035e2e3 cachingdns-edge packagevariant-11 false Draft us-central1-edge-02
us-central1-edge-02-3784c881bbf0490480786935fa4524ade10ee087 cachingdns-edge packagevariant-28 false Draft us-central1-edge-02
us-central1-edge-02-395852bc65dd70791213a6b6534973bcd1d8d8cd cachingdns-edge packagevariant-24 false Draft us-central1-edge-02
us-central1-edge-02-416da315c383ae26e33d3139e7dabd335fe04ecf cachingdns-edge packagevariant-4 false Draft us-central1-edge-02
us-central1-edge-02-487f5fd64471a08a0817f37843a5d9477a9fe33e cachingdns-edge packagevariant-19 false Draft us-central1-edge-02
us-central1-edge-02-49f0e8b9458b2f995544df5a4732dc0d95ff066a cachingdns-edge packagevariant-30 false Draft us-central1-edge-02
us-central1-edge-02-4bd6f30b30795031caff80f0e2fdc25a573701b2 cachingdns-edge packagevariant-22 false Draft us-central1-edge-02
us-central1-edge-02-5624c548be4419e5e50820f346fe4b855b0b6691 cachingdns-edge packagevariant-15 false Draft us-central1-edge-02
us-central1-edge-02-5f6a1c61c1e296922b457f099cbd893c1a1d9c04 cachingdns-edge packagevariant-21 false Draft us-central1-edge-02
us-central1-edge-02-64641a2a553e9c40af9aa21fdfbf4cf531c7349f cachingdns-edge packagevariant-2 false Draft us-central1-edge-02
us-central1-edge-02-66ebded096fe9bc454b12b0ffed761f98f3a628f cachingdns-edge packagevariant-32 false Draft us-central1-edge-02
us-central1-edge-02-6e4f72848e5d0b609e72012161fa245b0312119d cachingdns-edge packagevariant-8 false Draft us-central1-edge-02
us-central1-edge-02-6e965eca522fd0a47eec31778b6736d70ef4c0a8 cachingdns-edge packagevariant-5 false Draft us-central1-edge-02
us-central1-edge-02-7402304235c26cfccce8c78510dbcefd90a2e41d cachingdns-edge packagevariant-14 false Draft us-central1-edge-02
us-central1-edge-02-78e6ec06c813b6e5210015e7525df1546d122259 cachingdns-edge packagevariant-1 false Draft us-central1-edge-02
us-central1-edge-02-84ac1ac174d3be6eb56f8954660663b0250d0683 cachingdns-edge packagevariant-7 false Draft us-central1-edge-02
us-central1-edge-02-8eeb3ea6942da97ac354cbd71114f8c8cccfa2a6 cachingdns-edge packagevariant-25 false Draft us-central1-edge-02
us-central1-edge-02-8f194f5ae63dac4946be76c15d69a1384980a061 cachingdns-edge packagevariant-26 false Draft us-central1-edge-02
us-central1-edge-02-9ccf6de38e495c327372a04dabf63e3276a716a3 cachingdns-edge packagevariant-29 false Draft us-central1-edge-02
us-central1-edge-02-a4f829aa9b7ed8816d327b1503c7336210cb1e13 cachingdns-edge packagevariant-12 false Draft us-central1-edge-02
us-central1-edge-02-abcc12021c54e5a19d36e9be2fcda2e5558b063d cachingdns-edge packagevariant-3 false Draft us-central1-edge-02
us-central1-edge-02-b9da53a7ba942accc51520d5ad715a3459bf7d3e cachingdns-edge packagevariant-18 false Draft us-central1-edge-02
us-central1-edge-02-bc97ce6211f08e395e71e853fc2fdf57774da6f5 cachingdns-edge packagevariant-10 false Draft us-central1-edge-02
us-central1-edge-02-c5d5e4d1d358cab4ea6aab52151ec2626bc3191c cachingdns-edge packagevariant-16 false Draft us-central1-edge-02
us-central1-edge-02-c8480450e11ed9d3eb82270606665ea0493de856 cachingdns-edge packagevariant-23 false Draft us-central1-edge-02
us-central1-edge-02-d33d3cd9b59cbb3e959a03404a156b172f55d12d cachingdns-edge packagevariant-13 false Draft us-central1-edge-02
us-central1-edge-02-dbfc104305f952056bc2037d7ce0f8363d46baa6 cachingdns-edge packagevariant-9 false Draft us-central1-edge-02
us-central1-edge-02-eec3ede48babddb5d98fad34d4422f0325e947bd cachingdns-edge packagevariant-27 false Draft us-central1-edge-02
us-central1-edge-02-ef67e0e9ec797aa76086e1bae7eac958efe3088d cachingdns-edge packagevariant-17 false Draft us-central1-edge-02
us-central1-edge-02-efc16abcf6b52ac234170799af0b30356024daca cachingdns-edge packagevariant-31 false Draft us-central1-edge-02
us-central1-edge-02-f1e71381e54f7850a73a43e538f8c47ad23c89c7 cachingdns-edge packagevariant-34 false Draft us-central1-edge-02
us-central1-edge-02-fadd99b4f301abf29dd1cd7fd46e1e2121b872d6 cachingdns-edge packagevariant-20 false Draft us-central1-edge-02
us-central1-edge-03-e01e67636b8d845b321a650a3423d07b9881b97c cachingdns-edge packagevariant-1 false Draft us-central1-edge-03
us-central1-edge-04-fc22f957113a122cf04158b8dfe079c9f7503e99 cachingdns-edge packagevariant-1 false Draft us-central1-edge-04
us-central1-regional-92794a396633223eff36ed41701210b6e319d5f5 cachingdns-distribution packagevariant-1 false Draft us-central1-regional
us-central2-regional-322bcb84ae8a8a6ddd40513f3a2bb55f99b3f75a cachingdns-distribution packagevariant-1 false Draft us-central2-regional
us-east1-edge-01-459f97554e7a1551072260a0bc616f6c4e9f7968 cachingdns-edge packagevariant-1 false Draft us-east1-edge-01
us-east1-edge-02-2b0efaed7ac3998524a8b01e968a5c29843d5cf4 cachingdns-edge packagevariant-1 false Draft us-east1-edge-02
us-east1-regional-01-72d397520036dd95fd49d60589b7a76edab5709e cachingdns-distribution packagevariant-1 false Draft us-east1-regional-01
us-east1-regional-02-bed387c8e383e4c20355bcb496811b300b3da653 cachingdns-distribution packagevariant-1 false Draft us-east1-regional-02
us-east2-regional-183271f74d1feed2ffe5058c938945067b2c8559 cachingdns-distribution packagevariant-1 false Draft us-east2-regional
The us-central1-edge-02 variant has gone crazy!
This is in the logs a lot:
I0313 18:33:54.170521 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:34:53.380233 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:35:53.774376 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:36:55.084324 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:37:54.785738 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:38:55.879202 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:39:55.783493 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:40:52.972026 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:41:57.681830 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:42:56.670062 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:43:57.176065 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:44:52.470967 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:45:57.877849 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:46:55.880729 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0313 18:47:52.878709 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
Information
Steps to reproduce the behavior
I'll work on a cleaner repro.
Controller log. Seeing this a lot:
I0314 17:06:26.474898 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
E0314 17:07:00.871929 1 controller.go:326] "msg"="Reconciler error" "error"="Timeout: request did not complete within requested timeout - context deadline exceeded" "PackageVariant"={"name":"cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7","namespace":"default"} "controller"="packagevariant" "controllerGroup"="config.porch.kpt.dev" "controllerKind"="PackageVariant" "name"="cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" "namespace"="default" "reconcileID"="63524b1d-6e1d-423f-87f9-2f37853e695d"
I0314 17:07:00.879073 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
E0314 17:07:35.081099 1 controller.go:326] "msg"="Reconciler error" "error"="Timeout: request did not complete within requested timeout - context deadline exceeded" "PackageVariant"={"name":"cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7","namespace":"default"} "controller"="packagevariant" "controllerGroup"="config.porch.kpt.dev" "controllerKind"="PackageVariant" "name"="cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" "namespace"="default" "reconcileID"="11b1575b-25da-4a43-8ffa-b1ffee71b351"
I0314 17:08:23.772097 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
E0314 17:08:58.073898 1 controller.go:326] "msg"="Reconciler error" "error"="Timeout: request did not complete within requested timeout - context deadline exceeded" "PackageVariant"={"name":"cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7","namespace":"default"} "controller"="packagevariant" "controllerGroup"="config.porch.kpt.dev" "controllerKind"="PackageVariant" "name"="cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" "namespace"="default" "reconcileID"="b07c51c5-fa62-47ef-a7fd-4d055616c857"
I0314 17:08:58.081976 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
E0314 17:09:32.371065 1 controller.go:326] "msg"="Reconciler error" "error"="Timeout: request did not complete within requested timeout - context deadline exceeded" "PackageVariant"={"name":"cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7","namespace":"default"} "controller"="packagevariant" "controllerGroup"="config.porch.kpt.dev" "controllerKind"="PackageVariant" "name"="cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" "namespace"="default" "reconcileID"="62351605-0d6f-47a3-b804-6c56462c06c1"
This is so weird. I deleted all the packagevariantsets; all the packagevariants were deleted (see though #3878). I manually deleted all the lingering branches for "edge-02" (see #3882). I used kpt alpha rpkg del to delete the remaining lingering Drafts in the other repos. So, in theory we are back to original state. But, I redeploy the PVSs, and I am seeing the same thing again, with the same repo:
us-central1-edge-01-fc057482816768663fef81c4e5252856a632751d cachingdns-edge packagevariant-1 false Draft us-central1-edge-01
us-central1-edge-02-037d2e76d7b3cfce0670a50cc580c4d3b7b950b4 cachingdns-edge packagevariant-203 false Draft us-central1-edge-02
us-central1-edge-02-3a293d54ed1fc31baf0751cbbf8e3010509fc82a cachingdns-edge packagevariant-202 false Draft us-central1-edge-02
us-central1-edge-02-49585d06272feccda423ee21e0c8314fb8ea17d0 cachingdns-edge packagevariant-208 false Draft us-central1-edge-02
us-central1-edge-02-50cb78cc4cffd97f4789c06341bee481d3b050d4 cachingdns-edge packagevariant-200 false Draft us-central1-edge-02
us-central1-edge-02-698e57415b142775421c1a9e0f1136feebf93ba0 cachingdns-edge packagevariant-206 false Draft us-central1-edge-02
us-central1-edge-02-749452573d6327c1e722e2ffe40e00077e1e68b2 cachingdns-edge packagevariant-201 false Draft us-central1-edge-02
us-central1-edge-02-d27235098b919b529879725e6bf345382da7f888 cachingdns-edge packagevariant-210 false Draft us-central1-edge-02
us-central1-edge-02-e1ab6665a67f553a9eeb96942de5a099c2d5c677 cachingdns-edge packagevariant-211 false Draft us-central1-edge-02
us-central1-edge-02-f68df374d0e36061b345bdf89f98a43fcc72b2d7 cachingdns-edge packagevariant-207 false Draft us-central1-edge-02
us-central1-edge-02-f9c0f18c7bacd9a5c8ae05442c07ff6d93fea57b cachingdns-edge packagevariant-205 false Draft us-central1-edge-02
us-central1-edge-02-fc0ea9d97bf4e174d7e1b3b427962c8b472d20c0 cachingdns-edge packagevariant-204 false Draft us-central1-edge-02
us-central1-edge-02-fde0df04692f2ad2926ead7e72ee17561abbc1cd cachingdns-edge packagevariant-209 false Draft us-central1-edge-02
More info:
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ k get packagerevision us-central1-edge-02-3a293d54ed1fc31baf0751cbbf8e3010509fc82a -o yaml
apiVersion: porch.kpt.dev/v1alpha1
kind: PackageRevision
metadata:
creationTimestamp: "2023-03-14T18:48:02Z"
name: us-central1-edge-02-3a293d54ed1fc31baf0751cbbf8e3010509fc82a
namespace: default
resourceVersion: 854df9b1b3ffb03d18a93ea235fde92fb0a06f63
uid: uid:cachingdns-edge:packagevariant-202
spec:
lifecycle: Draft
packageName: cachingdns-edge
repository: us-central1-edge-02
tasks:
- clone:
upstreamRef:
upstreamRef:
name: nephio-testorg-packages-4ad103ac302c07b54dd709719f6ef500d64424d9
type: clone
- eval:
config: null
image: render
match: {}
type: eval
workspaceName: packagevariant-202
status:
deployment: true
publishTimestamp: null
upstreamLock:
git:
commit: 3c8d8c0fb2081375ac31c01db07563dff00dc190
directory: cachingdns-edge
ref: cachingdns-edge/v2
repo: https://github.com/johnbelamaric/nephio-testorg-packages.git
type: git
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ k get repository us-central1-edge-02 -o yaml
apiVersion: config.porch.kpt.dev/v1alpha1
kind: Repository
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"config.porch.kpt.dev/v1alpha1","kind":"Repository","metadata":{"annotations":{},"labels":{"kpt.dev/deployment-environment":"production","nephio.org/region":"us-central1","nephio.org/site":"us-central1-edge-02","nephio.org/site-type":"edge"},"name":"us-central1-edge-02","namespace":"default"},"spec":{"content":"Package","deployment":true,"description":"US Central 1 Edge Site 02 Cluster Workloads","git":{"branch":"main","createBranch":true,"directory":"/","repo":"https://github.com/nephio-test/jbelamaric-us-central1-edge-02.git","secretRef":{"name":"nephio-test-auth"}},"type":"git"}}
creationTimestamp: "2023-03-09T22:47:01Z"
generation: 2
labels:
kpt.dev/deployment-environment: production
nephio.org/region: us-central1
nephio.org/site: us-central1-edge-02
nephio.org/site-type: edge
name: us-central1-edge-02
namespace: default
resourceVersion: "5635626"
uid: e7409f75-997e-41aa-bef2-33b324fdd341
spec:
content: Package
deployment: true
description: US Central 1 Edge Site 02 Cluster Workloads
git:
branch: main
createBranch: true
directory: /
repo: https://github.com/nephio-test/jbelamaric-us-central1-edge-02.git
secretRef:
name: nephio-test-auth
type: git
status:
conditions:
- lastTransitionTime: "2023-03-09T23:06:36Z"
message: ""
observedGeneration: 2
reason: Ready
status: "True"
type: Ready
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ k get packagevariants
NAME AGE
cachingdns-distribution-1e718cfbdb754e386474da9aa91bd159ed27c6d7 16m
cachingdns-distribution-52cb15c06dc9252e966897bde315bc6b2e06af4f 16m
cachingdns-distribution-54154c218b775b84e280258f12f684c813e76c49 16m
cachingdns-distribution-aa0bd84da557680b052811b54e528625c13907cf 16m
cachingdns-distribution-d1579cce150353b437bdaa9845702b33f0900cdc 16m
cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7 16m
cachingdns-edge-3cc7f3a7d7f8d27acc6375cba42b5cc168d70fa0 16m
cachingdns-edge-44f604043046c915ea89667128b276b9c03211b7 16m
cachingdns-edge-596ce7b3b682f6eb284d7c1aec74aaaf179ad834 16m
cachingdns-edge-6b3bac48e67223051e4e47b5c52095eda9c1ee35 16m
cachingdns-edge-edf34f330bd64333cb936090fb4682e1d84f0159 16m
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$ k get packagevariant cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7 -o yaml
apiVersion: config.porch.kpt.dev/v1alpha1
kind: PackageVariant
metadata:
creationTimestamp: "2023-03-14T18:45:45Z"
finalizers:
- config.porch.kpt.dev/packagevariants
generation: 1
labels:
config.porch.kpt.dev/packagevariantset: 12a1914a-69e3-4050-acef-3bb97dc2683e
name: cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7
namespace: default
ownerReferences:
- apiVersion: config.porch.kpt.dev/v1alpha1
controller: true
kind: PackageVariantSet
name: cachingdns-edge
uid: 12a1914a-69e3-4050-acef-3bb97dc2683e
resourceVersion: "7160488"
uid: 8baa9b23-2511-4128-bbd5-2f659caa2943
spec:
adoptionPolicy: adoptNone
deletionPolicy: delete
downstream:
package: cachingdns-edge
repo: us-central1-edge-02
upstream:
package: cachingdns-edge
repo: nephio-testorg-packages
revision: v2
jbelamaric@jbelamaric:~/proj/tmp/cachingdns-topology$
From the logs, I see this over and over:
I0314 18:53:53.720819 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:54:51.903654 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:55:52.123161 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:57:10.288456 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:58:07.186308 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:58:52.597565 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
I0314 18:59:56.185249 1 packagevariant_controller.go:253] package variant "cachingdns-edge-2795d42dbf61192c4c66795f7f35fe2589106ad7" is creating package revision ""
This is from the line:
klog.Infoln(fmt.Sprintf("package variant %q is creating package revision %q", pv.Name, newPR.Name))
So, @natasha41575, we are not setting newPR.Name when contructing the PackageRevision. Is this intentional? Is the idea that Porch will create a name for us? That's fine - but for some reason, just with this package, even though there doesn't seem to be an error, we keep creating new PRs.
Ok, so this may be part of the issue. The PackageRevision is not created with the ownerRef. Look at these two PRs, one is for "edge-02" and one is not:
jbelamaric@jbelamaric:~/proj/gh/johnbelamaric/kpt/porch/controllers$ k get packagerevisions us-central1-edge-01-fc057482816768663fef81c4e5252856a632751d us-central1-edge-02-0359e96849f5e498f5bf529bfa799f027596167b -o yaml
apiVersion: v1
items:
- apiVersion: porch.kpt.dev/v1alpha1
kind: PackageRevision
metadata:
creationTimestamp: "2023-03-14T18:47:28Z"
name: us-central1-edge-01-fc057482816768663fef81c4e5252856a632751d
namespace: default
ownerReferences:
- apiVersion: config.porch.kpt.dev/v1alpha1
controller: true
kind: PackageVariant
name: cachingdns-edge-596ce7b3b682f6eb284d7c1aec74aaaf179ad834
uid: cb41117f-ae87-4b5f-8606-bc7e7552bbb9
resourceVersion: 30952d6417924df47c6aa9f914062f998ec7cd4b
uid: uid:cachingdns-edge:packagevariant-1
spec:
lifecycle: Draft
packageName: cachingdns-edge
repository: us-central1-edge-01
tasks:
- clone:
upstreamRef:
upstreamRef:
name: nephio-testorg-packages-4ad103ac302c07b54dd709719f6ef500d64424d9
type: clone
- eval:
config: null
image: render
match: {}
type: eval
workspaceName: packagevariant-1
status:
deployment: true
publishTimestamp: null
upstreamLock:
git:
commit: 3c8d8c0fb2081375ac31c01db07563dff00dc190
directory: cachingdns-edge
ref: cachingdns-edge/v2
repo: https://github.com/johnbelamaric/nephio-testorg-packages.git
type: git
- apiVersion: porch.kpt.dev/v1alpha1
kind: PackageRevision
metadata:
creationTimestamp: "2023-03-14T19:23:39Z"
name: us-central1-edge-02-0359e96849f5e498f5bf529bfa799f027596167b
namespace: default
resourceVersion: 044fd1e1c1478e47d82c147ee0ebf367b1ee8ea7
uid: uid:cachingdns-edge:packagevariant-237
spec:
lifecycle: Draft
packageName: cachingdns-edge
repository: us-central1-edge-02
tasks:
- clone:
upstreamRef:
upstreamRef:
name: nephio-testorg-packages-4ad103ac302c07b54dd709719f6ef500d64424d9
type: clone
- eval:
config: null
image: render
match: {}
type: eval
workspaceName: packagevariant-237
status:
deployment: true
publishTimestamp: null
upstreamLock:
git:
commit: 3c8d8c0fb2081375ac31c01db07563dff00dc190
directory: cachingdns-edge
ref: cachingdns-edge/v2
repo: https://github.com/johnbelamaric/nephio-testorg-packages.git
type: git
kind: List
metadata:
resourceVersion: ""
jbelamaric@jbelamaric:~/proj/gh/johnbelamaric/kpt/porch/controllers$
The one that is working has the ownerRef set, the other doesn't. I have no idea how that is happening, though, since it is set here:
https://github.com/GoogleContainerTools/kpt/blob/833194ed14b5a9cd31bc6ccab50a6c7a5e809100/porch/controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go#L230
Actually, one of the few dozen entries for edge-02 does have an ownerRef. Huh.
I'm trying to understand why this is happening just with us-central1-edge-02. Is there anything obviously different about that one vs all your other ones?
Also, could you share what your PackageVariantSets look like? I'm wondering if it could possibly be creating multiple PackageVariants that are causing strange interactions with each other. In any case it will help me repro this. I assume the repo you are using is https://github.com/johnbelamaric/nephio-testorg-deploy, is that right?
No on that repo, because of the other bug. I am using http://github.com/nephio-test
I am kpt live apply'ing this package: https://github.com/johnbelamaric/nephio-testorg-packages/tree/main/cachingdns-topology
Repos are setup like this the yamls here, using the repos.sh script in there (see, they no longer point to the directories): https://github.com/johnbelamaric/nephio-testorg-deploy
Oddly I am not seeing the same behavior when I run the controller locally.
The other weird thing as I mentioned before is that the UI does not show the bogus package revisions, only the CLI and of course looking directly at the repo.
So, after restarting the porch-controllers and the porch-server having restarted itself:
- I haven't seen any references to the old, deleted nephio-testorg-deploy repository
- I haven't seen any new bogus packages be created
This makes me think that somehow the porch-server state got confused due to all the previous errors, and perhaps this is the source of the issue. I still can't explain it. But I will watch to see if there are more issues.
So as I said offline, since this is (hopefully) an uncommon issue that will be difficult to repro, I think we can put it in the backlog for now. It's possible that porch was confused by all the previous errors, but I'm still not sure how that would affect the PackageVariant's ownerrefs and detection of existing packages. If we see any similar issues, we should investigate further.