milvus-operator icon indicating copy to clipboard operation
milvus-operator copied to clipboard

cant install by default cluster yaml

Open Sallery-X opened this issue 8 months ago • 6 comments

operator version: 1.2.4

default cluster yaml:

This is a sample to deploy a milvus cluster in milvus-operator's default configurations.

apiVersion: milvus.io/v1beta1 kind: Milvus metadata: name: my-release labels: app: milvus spec: mode: cluster dependencies: {} components: {} config: {}

operator error log: {"level":"error","ts":"2025-04-15T06:22:23Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"my-release","namespace":"milvus-test"},"namespace":"milvus-test","name":"my-release","reconcileID":"8b70c32f-4b43-4f52-be80-667daded46d7","error":"Milvus.milvus.io "my-release" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"}

Sallery-X avatar Apr 15 '25 06:04 Sallery-X

Hi @Sallery-X, I failed to reproduce it in our environment. What's your Kubernetes version? Could you also provide full logs of milvus-operator?

haorenfsa avatar Apr 15 '25 07:04 haorenfsa

Hi @Sallery-X, I failed to reproduce it in our environment. What's your Kubernetes version? Could you also provide full logs of milvus-operator?

may I have your wechat? my: 17682313239

k8s version: 1.19

full log: version: 1.2.4 milvus-helm version: milvus-4.2.40 {"level":"info","ts":"2025-04-15T07:41:34Z","logger":"controller","msg":"enable webhook","enable":false} {"level":"info","ts":"2025-04-15T07:41:34Z","logger":"setup","msg":"starting manager","version":"1.2.4","milvus-helm version":"milvus-4.2.40"} {"level":"info","ts":"2025-04-15T07:41:34Z","msg":"starting server","name":"health probe","addr":"[::]:8081"} {"level":"info","ts":"2025-04-15T07:41:34Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"} I0415 07:41:34.473970 1 leaderelection.go:254] attempting to acquire leader lease milvus-operator/71808ec5.milvus.io... {"level":"info","ts":"2025-04-15T07:41:34Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":":8080","secure":false} I0415 07:41:50.534777 1 leaderelection.go:268] successfully acquired lease milvus-operator/71808ec5.milvus.io {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting EventSource","controller":"milvusupgrade","controllerGroup":"milvus.io","controllerKind":"MilvusUpgrade","source":"kind source: *v1beta1.MilvusUpgrade"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting EventSource","controller":"milvusupgrade","controllerGroup":"milvus.io","controllerKind":"MilvusUpgrade","source":"kind source: *v1.Pod"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting Controller","controller":"milvusupgrade","controllerGroup":"milvus.io","controllerKind":"MilvusUpgrade"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting EventSource","controller":"milvuscluster","controllerGroup":"milvus.io","controllerKind":"MilvusCluster","source":"kind source: *v1alpha1.MilvusCluster"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting Controller","controller":"milvuscluster","controllerGroup":"milvus.io","controllerKind":"MilvusCluster"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting EventSource","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","source":"kind source: *v1beta1.Milvus"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting Controller","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus"} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting workers","controller":"milvuscluster","controllerGroup":"milvus.io","controllerKind":"MilvusCluster","worker count":1} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting workers","controller":"milvusupgrade","controllerGroup":"milvus.io","controllerKind":"MilvusUpgrade","worker count":1} {"level":"info","ts":"2025-04-15T07:41:50Z","msg":"Starting workers","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","worker count":10} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"setup loop","func":"syncUnealthyOrUpdating-fm","interval":"30s"} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"setup loop","func":"syncHealthyUpdated-fm","interval":"1m0s"} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"syncUnealthyOrUpdating start","time":"2025-04-15T07:41:53Z"} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"syncHealthyUpdated start","time":"2025-04-15T07:41:53Z"} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"setup loop","func":"updateMetrics-fm","interval":"30s"} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"syncHealthyUpdated end","duration":0.001073145} {"level":"info","ts":"2025-04-15T07:41:53Z","logger":"controller.status-syncer","msg":"syncUnealthyOrUpdating end","duration":0.001122789} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"4ab58499-2ee7-4b90-8e35-ca6a0e846926","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"76227207-7b3c-473e-a832-0c174ce3e842","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"009ab5d4-cc2e-4f5a-9d1e-1e12a98b2698","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"0fea6147-94dd-4f9c-9f50-16af8c7615c8","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"0adfef71-7b12-4c24-b15f-39a7cf243f5f","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"bb7843ae-c8ef-43c4-9094-bad7ecc208ec","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:53Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"28336d91-66ac-42d4-a7ba-8ba88ced6c85","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:54Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"f3bc16ad-2b53-41c5-b029-92052f83e602","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:54Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"0aa42ac3-3ce9-4f8f-a135-11bc72329a38","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:56Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"f2d05c31-a348-41f9-b6d2-c65ec8ab8e0a","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:41:58Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"ebe0c955-40f0-430a-924e-c6693eee6cf4","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"} {"level":"error","ts":"2025-04-15T07:42:03Z","msg":"Reconciler error","controller":"milvus","controllerGroup":"milvus.io","controllerKind":"Milvus","Milvus":{"name":"milvus-test","namespace":"milvus-test"},"namespace":"milvus-test","name":"milvus-test","reconcileID":"5615a259-7508-4173-b16a-82da1ffc2f23","error":"Milvus.milvus.io "milvus-test" is invalid: spec.dependencies.etcd.endpoints: Invalid value: "null": spec.dependencies.etcd.endpoints in body must be of type array: "null"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"}

Sallery-X avatar Apr 15 '25 07:04 Sallery-X

@haorenfsa thx a lot

Sallery-X avatar Apr 15 '25 07:04 Sallery-X

@haorenfsa I add “ endpoints: []” in demo yaml error disappear dependencies: etcd: endpoints: []

Sallery-X avatar Apr 15 '25 08:04 Sallery-X

@Sallery-X Seems to be a backward compatity problem for earlier kubernetes version.

haorenfsa avatar Apr 15 '25 08:04 haorenfsa

@haorenfsa I add “ endpoints: []” in demo yaml error disappear dependencies: etcd: endpoints: []

yes, that's a quick walk around.

haorenfsa avatar Apr 15 '25 08:04 haorenfsa