libiio icon indicating copy to clipboard operation
libiio copied to clipboard

DMABUF API v3

Open pcercuei opened this issue 1 year ago • 7 comments

Follow-up of https://github.com/analogdevicesinc/libiio/pull/928

These commits will modify how the local-DMABUF interface works, because the interface itself in the kernel changed (as it is not yet upstream).

The local backend will now make use of the "udmabuf" kernel driver, which was added in 5.19, to create DMABUF objects from memfd buffers. These DMABUFs can then be attached to the IIO device and/or other devices from other subsystems, and memory-mapped to access the data.

IIOD has also been modified to support enqueueing DMABUFs to the USB (functionfs) interface, and will support passing the buffers between IIO and USB in a zero-copy fashion.

From a Libiio API standpoint, some changes were introduced:

  • iio_block_enqueue will support partial transfers on RX buffers as well;
  • Setting bytes_used=0 in iio_block_enqueue will default to transferring the full buffer;
  • New API function iio_block_disable_cpu_access, to disable CPU synchronization with the buffers when doing zero-copy;
  • New API function iio_block_get_dmabuf_fd which can be used when doing zero-copy as well.

pcercuei avatar Aug 23 '23 08:08 pcercuei

Do we need some CMake and ifdef so that things still build on old kernels (pre 5.19?)

I think that is what is causing the centos 7 fails.

/__w/1/s/local-dmabuf.c: In function 'local_create_dmabuf':
/__w/1/s/local-dmabuf.c:100:56: error: 'MFD_ALLOW_SEALING' undeclared (first use in this function)
  memfd = syscall(__NR_memfd_create, "/libiio-udmabuf", MFD_ALLOW_SEALING);
                                                        ^
/__w/1/s/local-dmabuf.c:100:56: note: each undeclared identifier is reported only once for each function it appears in
/__w/1/s/local-dmabuf.c:106:19: error: 'F_ADD_SEALS' undeclared (first use in this function)
  if (fcntl(memfd, F_ADD_SEALS, F_SEAL_SHRINK) < 0) {
                   ^
/__w/1/s/local-dmabuf.c:106:32: error: 'F_SEAL_SHRINK' undeclared (first use in this function)
  if (fcntl(memfd, F_ADD_SEALS, F_SEAL_SHRINK) < 0) {
                                ^
make[2]: *** [CMakeFiles/iio.dir/local-dmabuf.c.o] Error 1
make[1]: *** [CMakeFiles/iio.dir/all] Error 2
make: *** [all] Error 2

rgetz avatar Aug 28 '23 12:08 rgetz

Updated the PR; it now uses dma-heaps instead of udmabuf to create the DMABUF object.

pcercuei avatar Jan 24 '24 14:01 pcercuei

Rebased and added a fix so that the DMABUFs are correctly detached from USB after #1136.

pcercuei avatar Jan 30 '24 12:01 pcercuei

@mhennerich reminder to merge this :)

Note that I cannot update the branch anymore as I'm not in ADI's organization, so I cannot rebase it.

pcercuei avatar Jul 23 '24 08:07 pcercuei

Hi Paul,

@nunojsa is in progress testing it on a backported 6.6 Linux kernel with DMABUF. We want to test it once more before merging it. ...Hopefully done by the end of the week in case testing doesn't exhibit any problems.

-Michael

mhennerich avatar Jul 23 '24 09:07 mhennerich

Nice! @nunojsa don't hesitate to ping me if you have problems or questions.

pcercuei avatar Jul 23 '24 09:07 pcercuei

@pcercuei,

I finally got to do some testing with this with an ADI updated kernel. Things look good and working. Just some hick ups on the MMAP interface (I guess you're more focused on testing the new API :)). While I'm eager to remove that code from ADI kernel, we still need to support it for at least one release.

Anyways, first patch can be squashed in the fixup (just put it like this so it's easier for you to see the change) and the second I guess it needs to be in it's own patch.

nunojsa avatar Aug 07 '24 08:08 nunojsa