[184430109158DF0F00] [192.168.1.200] [557.932] [system] [critical] Fatal error. Please report to developers. Log: 'PlgSrcCtrl' '87'

[BBwanaz]

I am running OAKD-POE inside a docker container. The app has been running smoothly for weeks but recently it started giving this error here:

camera       | [184430109158DF0F00] [192.168.1.200] [557.932] [system] [critical] Fatal error. Please report to developers. Log: 'PlgSrcCtrl' '87'
camera       | [2022-08-04 20:33:48.236] [warning] Monitor thread (device: 184430109158DF0F00 [192.168.1.200]) - ping was missed, closing the device connection
camera       | Traceback (most recent call last):
camera       |   File "/computer-vision/training/record.py", line 70, in <module>
camera       |     run_recorder()
camera       |   File "/computer-vision/training/record.py", line 54, in run_recorder
camera       |     inRgb = qRgb.get()
camera       | RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'

Furthermore, specifying the IP of the camera using: device_info = dai.DeviceInfo(ip) is returning "no device with such info" however, when I run it to auto find the device on the network, the finds the device and when I print the device info, it shows the same IP address.

Further Details:

• I am pulling the luxonis/depthai-library docker image like this:

FROM luxonis/depthai-library

COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt

ADD . /computer-vision

WORKDIR /computer-vision

ENV PYTHONPATH "${PYTHONPATH}:/computer-vision"

and then running the container using docker-compose:

version: '2.4'
services:
  cv-camera:
    container_name: camera
    build:
      context: ./
      dockerfile: docker/Dockerfile.training
  
    privileged: true
    device_cgroup_rules:
      - 'c 189:* rmw'
    volumes:
        - '/dev/bus/usb:/dev/bus/usb'
        - '/tmp/.X11-unix:/tmp/.X11-unix'
    environment:
        - DISPLAY=$DISPLAY
    
    command: python3 training/record.py

    network_mode: host

    restart: unless-stopped

The docker-compose basically starts up the container and runs the script python3 record.py which creates the pipeline, gets rgb data and encodes in into .h256 much like the ColorCamera.

#!/usr/bin/env python3
from distutils.command.upload import upload
import depthai as dai
import json
from datetime import date, datetime
from camera.pipeline import Pipeline
from camera.controls import Controls
from uploader import Uploader

DEFAULT_EXP_TIME = 20000
DEFAULT_SENS_ISO = 800
DEFAULT_FOCUS = 150
DEFAULT_BRIGHTNESS = 0
DEFAULT_LENGTH = 10

# Connect to device and start pipeline
pipe = Pipeline()
uploader = Uploader()

with open('config.json') as f:
    config = json.load(f)

autoFocus = config.get("autoFocus", "Fasle") in ["true", "True"]
brightness = int(config.get('brightness', DEFAULT_BRIGHTNESS))
expTime = int(config.get('expTime', DEFAULT_EXP_TIME))
sensIso = int(config.get('sensIso', DEFAULT_SENS_ISO))
focus = int(config.get('focus', DEFAULT_FOCUS))
vidLength = float(config.get("vidLength", DEFAULT_LENGTH))
autoFind = config.get("autoFind", "Fasle") in ["true", "True"]
poe = config.get("POE", "Fasle") in ["true", "True"]
ip = config.get("cameraIp", None)

def run_recorder(device):
    print("Starting Camera...", flush=True)
    
    # Get queues
    qRgb = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)
    controlQueue = device.getInputQueue('control')
    camControl = Controls(controlQueue=controlQueue)
    video = device.getOutputQueue(name="h265", maxSize=30, blocking=False)

    if not autoFocus:
        camControl.set_brightness(brightness)
        camControl.set_exposure(expTime=expTime, sensIso=sensIso)
        camControl.set_focus(focus)

    while True:
        t1 = datetime.now()
       
        with open('video.h265', 'wb') as videoFile:
            print("Press Ctrl+C to stop encoding...", flush=True)
            try:
                while True:
                    inRgb = qRgb.get()
                    h265Packet = video.get()  # Blocking call, will wait until a new data has arrived
                    h265Packet.getData().tofile(videoFile)  # Appends the packet data to the opened file

                    if (datetime.now() - t1).seconds >= 60 * vidLength:
                        break
            except KeyboardInterrupt:
                # Keyboard interrupt (Ctrl + C) detected
                break
        uploader.upload_file()
    uploader.upload_file()
    #ffmpeg -framerate 30 -i video.h265 -c copy video.mp4"

if autoFind or not poe:
    with dai.Device(pipe.create_pipeline()) as device:
      #  print(device.getDeviceInfo(), flush=True)
        run_recorder(device)

else:
    device_info = dai.DeviceInfo(ip)
    with dai.Device(pipe.create_pipeline(), device_info) as device:
        print(device.getDeviceInfo(), flush=True)
        run_recorder(device)

[themarpe]

@BBwanaz Do you mind checking the version of base image you are using? It seems like an older version is being used in this case. Try using either: v2.17.1 or v2.17.2

[BBwanaz]

Hi @themarpe

both versions and I still get the same error.

[Luxonis-Brandon]

Sorry about the trouble @BBwanaz . We'll likely need to dig into firmware to see how/why this is happening and what it will take to fix this.

[themarpe]

@BBwanaz

The latest v2.17.3 should address the issues when connecting with specific IP.

The app has been running smoothly for weeks but recently it started giving this error here

One question on the issue - does this happen continuously or it happened once after one week long run? And afterwards after another 1 week run? In that case you may catch the exception thrown by the device and reconnect upon such even, and log the issue upwards to notify any consumers, etc...