liburing icon indicating copy to clipboard operation
liburing copied to clipboard

io_uring.h: Add `#define`s for enum values

Open CarterLi opened this issue 1 year ago • 1 comments

I followed the pattern of <sys/epoll.h> to add #defines for enum values in liburing/io_uring.h

It should be useful when experimenting the bleeding-edge new features, so that I can test for their existance and don't break users that still use the old liburing version.


git request-pull output:

The following changes since commit 8bbcf56cde6ca248e6f3508454b3bcef82596a7f:

  Improve debian build (2023-07-13 15:11:35 -0600)

are available in the Git repository at:

  [email protected]:CarterLi/liburing.git master

for you to fetch changes up to 39210538c5fd865bb413eca53a420e35abe6b619:

  io_uring.h: Add `#define`s for enum values (2023-07-16 21:47:09 +0800)

----------------------------------------------------------------
李通洲 (1):
      io_uring.h: Add `#define`s for enum values

 src/include/liburing/io_uring.h | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)


Click to show/hide pull request guidelines

Pull Request Guidelines

  1. To make everyone easily filter pull request from the email notification, use [GIT PULL] as a prefix in your PR title.
[GIT PULL] Your Pull Request Title
  1. Follow the commit message format rules below.
  2. Follow the Linux kernel coding style (see: https://github.com/torvalds/linux/blob/master/Documentation/process/coding-style.rst).

Commit message format rules:

  1. The first line is title (don't be more than 72 chars if possible).
  2. Then an empty line.
  3. Then a description (may be omitted for truly trivial changes).
  4. Then an empty line again (if it has a description).
  5. Then a Signed-off-by tag with your real name and email. For example:
Signed-off-by: Foo Bar <[email protected]>

The description should be word-wrapped at 72 chars. Some things should not be word-wrapped. They may be some kind of quoted text - long compiler error messages, oops reports, Link, etc. (things that have a certain specific format).

Note that all of this goes in the commit message, not in the pull request text. The pull request text should introduce what this pull request does, and each commit message should explain the rationale for why that particular change was made. The git tree is canonical source of truth, not github.

Each patch should do one thing, and one thing only. If you find yourself writing an explanation for why a patch is fixing multiple issues, that's a good indication that the change should be split into separate patches.

If the commit is a fix for an issue, add a Fixes tag with the issue URL.

Don't use GitHub anonymous email like this as the commit author:

[email protected]

Use a real email address!

Commit message example:

src/queue: don't flush SQ ring for new wait interface

If we have IORING_FEAT_EXT_ARG, then timeouts are done through the
syscall instead of by posting an internal timeout. This was done
to be both more efficient, but also to enable multi-threaded use
the wait side. If we touch the SQ state by flushing it, that isn't
safe without synchronization.

Fixes: https://github.com/axboe/liburing/issues/402
Signed-off-by: Jens Axboe <[email protected]>

By submitting this pull request, I acknowledge that:

  1. I have followed the above pull request guidelines.
  2. I have the rights to submit this work under the same license.
  3. I agree to a Developer Certificate of Origin (see https://developercertificate.org for more information).

CarterLi avatar Jul 16 '23 13:07 CarterLi

Retargeted to master

CarterLi avatar Jul 16 '23 13:07 CarterLi

@CarterLi This is really cool, it should also help with compatibility issues right? Like if someone is on Linux 6.5 and the library enum has IORING_OP_FUTEX_WAIT (6.7 feature) it should still compile.

p.s Think the patch needs to be updated.

YoSTEALTH avatar Apr 03 '24 02:04 YoSTEALTH

Yep agree, I do think this would be useful. @CarterLi would you mind updating it to the current tree?

axboe avatar Apr 03 '24 02:04 axboe

This didnt seem to get author's interests. Closing

CarterLi avatar Apr 03 '24 02:04 CarterLi

@CarterLi lol, author is busy with too many things, please be patience and author is interested now, so could you please update! Thank you very much :)

YoSTEALTH avatar Apr 03 '24 02:04 YoSTEALTH

@axboe what to do with this PR? It would have been an awesome feature for compatibility issues.

YoSTEALTH avatar Apr 14 '24 03:04 YoSTEALTH

Well, as the original author didn't want to get it updated, maybe someone else will. I'm not doing anything with it as-is.

axboe avatar Apr 14 '24 12:04 axboe

That's pretty horrible for readability and maintenance, I wonder if there are better ways of doing it.

isilence avatar Apr 14 '24 22:04 isilence

That's pretty horrible for readability and maintenance, I wonder if there are better ways of doing it.

@isilence I asked few people if there was a better way! they pretty much laughed at me ...

I am not sure if this actually fixes the compatibility issues that I am after though! People said that adding enum #define key value does not account for what kernel supports. Its going to add those enum regardless.

I think I am after something like what io_uring_probe provides but at compile time! Or maybe I should just compile the C liburing, and run probe, to see what supported before running my own compiler? dono...

YoSTEALTH avatar Apr 15 '24 02:04 YoSTEALTH

I think I am after something like what io_uring_probe provides but at compile time! Or maybe I should just compile the C liburing, and run probe, to see what supported before running my own compiler? dono...

But you can't do that at compile time, unless you're always compiling on the host that will run the application, as things may look very different once it's actually run. This is why runtime checking like io_uring_probe() exists. The alternative is to always hope features you need are there, and then deal with it if you get a failure using them.

The ifdef approach only solves one thing, and that's using newer features and still being able to compile. It doesn't solve the runtime side at all.

axboe avatar Apr 15 '24 14:04 axboe

fwiw, I totally get where it comes from, and there is a merit for those using a system installed liburing. At least they would be able to build their stuff without enforcing liburing version dependency. Otherwise would need ./configure scripting, which is even worse

isilence avatar Apr 15 '24 14:04 isilence

For sure, this is why I was willing to entertain this change. It solves the annoying issues if applications needing to do

#ifndef FOO
#define FOO something_that_is_in_newer_liburing
#endif

and basically provides that for the opcodes. But it obviously doesn't cover any runtime side checking, which there seems to be some confusion about here.

liburing is fully independent from the kernel in the sense that you should be able to run any version of either. The liburing installed tells you nothing of what's available on the kernel side, in either direction.

axboe avatar Apr 15 '24 14:04 axboe

This is what happening

using compiler (slow install):

  1. I include C liburing source code (so library is not dependent on whats installed in OS)
  2. run only ./configure and make (does not make install)
  3. run cython compiler that takes liburing.so and uses it for linking/symbol (don't really know) and finally creates my_module.so
  4. all files .c, .h, .so including liburing.so are removed, only my_module.so remains.
  5. finally in python you can do from my_module import my_function

using wheel (fast install. haven't done this, currently don't have virtual systems to test/compile with): a. compiling is done on developers build system b. .whl file is created c. people can use .whl file to install in seconds! d. there is no ./configure check or anything like that, you only find out error at runtime.

Currently on GitHub futex2 is not supported, even though its not the best way to go about it, maybe I could have used ./configure output to know that futex2 is not supported! I do:


    #ifndef FUTEX2_PRIVATE
        #define FUTEX2_PRIVATE  0
        // ...

And check in python runtime side


    if FUTEX2_PRIVATE:
        # do stuff ...

In theory I should be able to run io_uring_probe between step 2 & 3, and somehow pass that data into cython and have it only build what available on peoples system. This only works for compiler way, the wheel way its doesn't work.

So ya multiple problems to deal with...

YoSTEALTH avatar Apr 15 '24 22:04 YoSTEALTH