octopus icon indicating copy to clipboard operation
octopus copied to clipboard

devicelink should add log and retry if no reference was found in the same namespace

Open guangbochen opened this issue 4 years ago • 1 comments

Environment information:

  • Octopus version : master
  • Installer : all-in-one

Cluster information:

  • Cluster vendor : k3s
  • Machine type : vm
  • Kubernetes version (use kubectl version):
  • Docker version (use docker version):

Machine specification:

Role Number Specification (CPU/RAM)
master 1 4U8G
agent 2 4U8G

Steps to reproduce (least amount of steps as possible):

devicelink should add a log if no secret or configmap reference was found in the same namespace, also if adding the reference resource after maximum reconcile it will stop read the reference resource unless re-create the DL.

  1. deploy octopus with mqtt support and apply the dummy all-in-one.yaml
  2. run kubectl apply -f ./deploy/e2e/dl_specialdevice_with_mqtt.yaml to a new namespace and skip to create the secret certificate
  3. the living-room-fan DL is showing as healthy but the MQTT connection is failed due to empty certificate and limb show error log as below

Result:

2020-06-12T08:15:14.031Z	ERROR	controller.deviceLink	Unable to fetch the reference parameters of DeviceLink	{"deviceLink": "testx/living-room-fan", "error": "Secret \"living-room-fan-mqtt-ca\" not found"}
github.com/rancher/octopus/pkg/util/log/zap.(*logger).Error
	/Users/guangbochen/src/rancheredge/octopus/pkg/util/log/zap/logr.go:33
github.com/rancher/octopus/pkg/limb/controller.(*DeviceLinkReconciler).Reconcile
	/Users/guangbochen/src/rancheredge/octopus/pkg/limb/controller/devicelink.go:299
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88

devicelink should add a log and retry if no reference was found in the same namespace

Other details that may be helpful:

Except:

  1. add a log to the DL if no reference resource was found.
  2. add a log to the DL if maximum reconcile is reached for getting the reference resource(this is because we don't list watch those resources).

Related Issues:

guangbochen avatar Jun 12 '20 08:06 guangbochen

case one is resolved in https://github.com/cnrancher/octopus/pull/83

guangbochen avatar Jun 16 '20 07:06 guangbochen