Camera frame and encoder position get out of sync in high belt speed

[spoorgholi74]

Describe what you want to implement and what the issue & the steps to reproduce it are:

Hi, I have an architecture which a rotation encoder is connected to a belt and sends a signal every 10cm to a PLC which updates the PLC with the position of the encoder and also triggers the Basler camera to capture and image. With my PC I have a custom handshake with the PLC. Every rising edge of the encoder signal I read the conveyor position from the PLC and then in the same thread I read the framer from the camera and process the frame and metadata. Since everything runs in one loop sometimes the processing takes longer than the next rising edge of the encoder and the PC misses the read of the conveyor position and the camera frame.

However this should not be an issue since every time I must read the last frame from the camera using LatestImageOnly strategy and read the last value of the conveyor position. However, sometimes I can see that the last value of the encoder is read but the previously skipped frame is return from the camera.

So either the metadata is one tick behind or the image is one tick behind.

I don't know how this can happen. Maybe while grabbingresult the camera is still exposing image? And what is the best solution to fix this?an

Aquisition code

` def get_frame(self): camera_metadata = {} while True: time.sleep(0.01)

        grab_result, frame = False, False
        try:
            # Retrieves a grab result according to the grab strategy.
            # First param is the timeoutMs and the second is the handling method
            # Todo: Check the camera is not grabbing
            grab_result = self.cam.RetrieveResult(
                1000, pylon.TimeoutHandling_ThrowException
            )
            if not grab_result or not grab_result.GrabSucceeded():
                continue
            """
            The timestamps on Basler cameras have a resolution around 10ns per tick.
            Even if you have two cameras with synchronized clocks and both are
            triggered from the same signal small differences in the electrical
            circuits and your cable lengths will result in different timestamps.
            """
            try:
                img_captured_id = grab_result.GetID()
                img_retrived_nmbr = grab_result.GetImageNumber()
                time_stamp = grab_result.GetTimeStamp()
                n_skipped_images = grab_result.GetNumberOfSkippedImages()
                camera_metadata = {
                    "cam_capture_id": img_captured_id,
                    "cam_retrived_id": img_retrived_nmbr,
                    "cam_capture_time": time_stamp,
                    "cam_skipped_images": n_skipped_images
                }
                logger.debug(
                    "Basler fetched img_captured_id=%s , img_retrived_nmbr=%s and timestamp=%s and skipped_images=%s from the camera."
                    % (
                        img_captured_id,
                        img_retrived_nmbr,
                        time_stamp,
                        n_skipped_images,
                    )
                )
            except Exception as e:
                logger.error("Error reading metadata from camera %s" % e)
            # Todo: What is this locks and delay for
            with self.lock:
                self.set_user_output(True)  # Set UserOuput1 to "high"

            self.pool.submit(
                lambda: self.set_user_output(False, delay=0.1)
            )  # Restore flag after delay
            frame = self.converter.Convert(grab_result)
            frame = frame.GetArray()

        except Exception as e:
            logger.error("Error capturing frame from the camera --> %s" % e)
            raise

        finally:
            if grab_result:
                grab_result.Release()

        return frame, camera_metadata`

Is your camera operational in Basler pylon viewer on your platform

Yes

Hardware setup & camera model(s) used

Camera: Basler acA4096-11gc GigE PC: Ubuntu with GPu for image processing

Runtime information:

pypylon: 1.8.0
Python: 3.8

[HighImp]

Hi, Just a guess, but like you said, the camera can still exposure or transmitt Image n, while Trigger n from plc is processed by the PC, and read the last complete Image from Buffer, which is Image n-1.

May you can wait in your plc for one frame period after the camera trigger before informing the PC?

Best regards

[cpt-wojtech]

Hello @spoorgholi74 , when utilizing the LatestImageOnly strategy the buffer in the queue is overwritten immediately if a new buffer is available. So basically there is no possibility that you read out the previous image if really a new image is available. That is why I think too it's an desync issue and your handshake with the PLC happens faster at some point while the camera hasn't delivered the image yet. This should happen more or less after a defined period of time (there is some variation because of possible jitter and transmission delays on the ethernet line) and after some more time the desync should get even bigger.

Please take into account the parameters ExposureTime and ResultingFramerate of the camera and calculate if the timings fit into the speed of you conveyor belt signal (some more info about acquisition timings here