edgetpu icon indicating copy to clipboard operation
edgetpu copied to clipboard

Run on Kubernetes cluster

Open IArchi opened this issue 2 years ago • 4 comments

Given the docker command is (and works perfectly):

sudo docker run -t -i --privileged -v /dev/bus/usb:/dev/bus/usb myCoralImage:latest

The kubernetes yaml file should looks like this:

	apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: cameraai
  labels:
	app.kubernetes.io/name: cameraai
	app.kubernetes.io/instance: cameraai
	app.kubernetes.io/version: "0.1"
spec:
  selector:
	matchLabels:
	  app.kubernetes.io/name: cameraai
	  app.kubernetes.io/instance: cameraai
  template:
	metadata:
	  labels:
		app.kubernetes.io/name: cameraai
		app.kubernetes.io/instance: cameraai
	spec:
	  containers:
		- name: camera-ai
		  image: "myCoralImage:latest"
		  imagePullPolicy: Always
		  securityContext:
			privileged: true
		  volumeMounts:
			- name: dev-usb
			  mountPath: /dev/bus/usb
	  volumes:
		- name: dev-usb
		  hostPath:
			path: /dev/bus/usb

But this does not work. The Coral Edge TPU (USB Dongle) is not detected by the pod.

Is there any parameter to add to make it works ?

Thanks

IArchi avatar Oct 11 '21 14:10 IArchi

can you try to follow this and see if it works..


apiVersion: v1
kind: Pod
metadata:
  name: http-echo-edgetpu-pod
  labels:
    app: http-echo
spec:
  containers:
  - name: http-echo
    image: adaptant/http-echo
    imagePullPolicy: IfNotPresent
    args: [ "-text", "hello from a node with an EdgeTPU" ]
    ports:
    - containerPort: 5678
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          # USB-attached Coral AI Accelerator
          - key: feature.node.kubernetes.io/usb-fe_1a6e_089a.present
            operator: In
            values: [ "true" ]
        - matchExpressions:
          # Coral Dev Board  
          - key: beta.devicetree.org/fsl-imx8mq-phanbell
            operator: In
            values: [ "1" ]

hjonnala avatar Oct 11 '21 15:10 hjonnala

dt-labeller seems to only be suitable for arm64. My server's architecture is amd64.

nodeSelector:
    kubernetes.io/arch: arm64

Running the DaemonSet or the NFD (even without the nodeSelector) failed:

panic: no valid device tree configuration found
goroutine 1 [running]:
main.main()
      /app/main.go:179 +0xd85

IArchi avatar Oct 12 '21 09:10 IArchi

I've tried to deploy a very basic pod on my Kubernetes cluster.

from pycoral.utils.edgetpu import list_edge_tpus
print(list_edge_tpus(), flush=True)

Running directly into a Docker container produces the following results: [{'type': 'usb', 'path': '/sys/bus/usb/devices/4-1'}]

While running in the Kubernetes pods produces: []

This seems Kubernetes has not mounted the USB dongle.

IArchi avatar Oct 14 '21 08:10 IArchi

Can you please try this and let us know if its working or not.

hjonnala avatar Oct 26 '21 15:10 hjonnala