bcm2835-unicam and cfe metadata byte alignment problem

[ArduCAM]

Describe the bug

We tested the IMX519 and found that metadata can be output normally on Pi4, but on Pi5, the cfe driver reported "Wrong metadata width/height/code". After a simple calculation, we found that it seems that the cfe driver requires the length of Metadata to be 64bit aligned. Any ideas?

Steps to reproduce the behaviour

Running IMX519 on Pi5 using libcamera

Device (s)

Other

System

cat /etc/rpi-issue Raspberry Pi reference 2023-10-10 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, fb56ad562991cf3ae5c96ab50983e1deeaefc7b6, stage4

vcgencmd version 2023/10/18 18:30:17 Copyright (c) 2012 Broadcom version c2da2ae7 (release) (embedded)

uname -a Linux raspberrypi 6.1.0-rpi6-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.58-1+rpt2 (2023-10-27) aarch64 GNU/Linux

Logs

[    8.020555] macb 1f00100000.ethernet: gem-ptp-timer ptp clock registered.
[    8.051440] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
[    9.914906] Bluetooth: RFCOMM TTY layer initialized
[    9.914919] Bluetooth: RFCOMM socket layer initialized
[    9.914929] Bluetooth: RFCOMM ver 1.11
[   12.126407] macb 1f00100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[   12.126427] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   34.596407] rp1-cfe 1f00128000.csi: WARNING: Wrong metadata width/height/code 17472x1 00007002 (remote pad set to 17460x1 00007002)

dmesg.log

Additional context

No response

[ArduCAM]

Hi @6by9

Any ideas? Should I modify the driver directly to align the length? I'm worried if other platforms won't match.

[6by9]

The metadata implementation using 2 pads is a totally downstream set of patches, therefore there are no other platforms that could be affected.

Mainline are still working on their solution, but it will be based around https://patchwork.linuxtv.org/project/linux-media/cover/[email protected]/ and the streams support that is now merged.

The buffer sizing is solely used for memory allocation. Unicam is very limited in options for handling the non-image data (it writes all incoming to the buffer with no further formatting), hence why it has always been width x 1 despite the incoming data potentially being spread over multiple CSI2 lines.

[ArduCAM]

The metadata implementation using 2 pads is a totally downstream set of patches, therefore there are no other platforms that could be affected.

Mainline are still working on their solution, but it will be based around https://patchwork.linuxtv.org/project/linux-media/cover/[email protected]/ and the streams support that is now merged.

The buffer sizing is solely used for memory allocation. Unicam is very limited in options for handling the non-image data (it writes all incoming to the buffer with no further formatting), hence why it has always been width x 1 despite the incoming data potentially being spread over multiple CSI2 lines.

Got it, I think I need to submit a pull request to fix it.

[ArduCAM]

Hi @6by9 We found a bad news. The data type of the PD data of IMX519 is 0x36, and it cannot be modified through registers. Is there any way for us to configure it? We looked at the code for the csi2 part and it seems that it is fixed at 0x12. (In addition, the data type of embedded data other than PD is 0x12)

[6by9]

Unicam just dumps all non-image data into the metadata buffer.

CFE does have full VC/DT filtering, and currently assumes that V4L2_META_FMT_SENSOR_DATA is 0x12 due to https://github.com/raspberrypi/linux/blob/rpi-6.1.y/drivers/media/platform/raspberrypi/rp1_cfe/cfe_fmts.h#L282-L286. The streams support that mainline are developing is the right way to solve this, as the PDAF data would come as a third stream from the sensor, and come out via a further /dev/videoN node. Pass as to when that will be sorted though.

[6by9]

@naushir As just discussed, not sure if there are other options available in the RP1 CSI2 receiver.

[naushir]

I'm reluctant to change data type handling right now given the testing we've done. Hopefully the streams api ought to allow different data ids to be muxed/demuxed from a channel correctly. This should be available to us very soon.

[ArduCAM]

I'm reluctant to change data type handling right now given the testing we've done. Hopefully the streams api ought to allow different data ids to be muxed/demuxed from a channel correctly. This should be available to us very soon.

Hi naushir, Is there any progress in this regard at the moment?

[naushir]

The V4l2 streams API is being worked on by the libcamera team. Work is progressing, but I don't have a timeline for when it'll be completed fully.

[ArduCAM]

The V4l2 streams API is being worked on by the libcamera team. Work is progressing, but I don't have a timeline for when it'll be completed fully.

Got it, thank you for the information.