Support IO_URING transport

[franz1981]

Fixes #4163

[franz1981]

I should add some check re domain socket support on JUnit tests

[vietj]

can you modify the github actions to run io_uring tests ?

[Reef3rm4n]

nice

[franz1981]

It seems that ubuntu-latest is hitting https://github.com/netty/netty-incubator-transport-io_uring/blob/07bf47cef7436d392623fef34eecf0cd0f9b88bb/README.md#i-tried-to-use-io_uring-but-got-javalangruntimeexception-failed-to-create-io_uring-ring-fd-cannot-allocate-memory

Meaning that:

• we should find a way to change mlock limits on it
• we run it through qemu or docker our tests

For the latter I need some assistance, because I believe it should be beneficial for all vertx tests, not just for these changes, to be a more reproducible build/tests, wdyt @vietj @tsegismont ?

[franz1981]

Following up: https://github.community/t/how-to-raise-ulimits-on-the-github-runner/181781

[franz1981]

@vietj time for some action :P https://github.com/netty/netty-incubator-transport-io_uring/pull/159/checks

It seems now we can make use of our CI to run io_uring tests yeeeeeeeeeee

[franz1981]

@vietj It seems I cannot make the io_uring part to complete because of intermittent epoll failures :O no idea what's going on

[franz1981]

This is the list of errors I'm getting by running separately the IO uring tests with the github CI:

Tests in error: 
  DatagramTest.testPauseResume:209->AsyncTestBase.await:121->AsyncTestBase.await:133 » IllegalState
  Http1xTest>AsyncTestBase.after:83->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  Http2MetricsTest>AsyncTestBase.after:83->HttpMetricsTestBase.tearDown:57->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  Http2Test>AsyncTestBase.after:83->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
io.vertx.core.http.WebSocketTest.testSharedServersRoundRobin(io.vertx.core.http.WebSocketTest)
  Run 1: WebSocketTest.testSharedServersRoundRobin:567->AsyncTestBase.assertTrue:378 null
  Run 2: WebSocketTest>AsyncTestBase.after:83->tearDown:118->VertxTestBase.tearDown:93->AsyncTestBase.tearDown:73->AsyncTestBase.afterAsyncTestBase:163 » IllegalState

io.vertx.core.http.WebSocketTest.testSharedServersRoundRobinWithOtherServerRunningOnDifferentPort(io.vertx.core.http.WebSocketTest)
  Run 1: WebSocketTest.testSharedServersRoundRobinWithOtherServerRunningOnDifferentPort:608->testSharedServersRoundRobin:567->AsyncTestBase.assertTrue:378 null
  Run 2: WebSocketTest>AsyncTestBase.after:83->tearDown:118->VertxTestBase.tearDown:93->AsyncTestBase.tearDown:73->AsyncTestBase.afterAsyncTestBase:163 » IllegalState

  NetTest>AsyncTestBase.after:83->tearDown:132->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState

Tests run: 5012, Failures: 0, Errors: 7, Skipped: 13

Investigating, but still seems a fairly low number (and flaky)

[franz1981]

New results after changing the UDP test:

Results :

Tests in error: 
  Http1xTest>AsyncTestBase.after:83->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  Http2MetricsTest>AsyncTestBase.after:83->HttpMetricsTestBase.tearDown:57->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  Http2MetricsTest>AsyncTestBase.after:83->HttpMetricsTestBase.tearDown:57->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  Http2Test>AsyncTestBase.after:83->HttpTestBase.tearDown:71->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState
  NetTest>AsyncTestBase.after:83->tearDown:132->VertxTestBase.tearDown:87->AsyncTestBase.close:662->AsyncTestBase.awaitLatch:598->AsyncTestBase.awaitLatch:602->AsyncTestBase.assertTrue:380->AsyncTestBase.handleThrowable:183 » IllegalState

Tests run: 5012, Failures: 0, Errors: 5, Skipped: 13

we're getting there :D

There's some problem while closing the Vert-x loop, checking

[vietj]

thanks @franz1981 we're almost there

[Sanne]

Awesome - I'm trying to regularly test Quarkus / Hibernate Reactive on a combination of different options, including io_uring, and if the patch could be included in vert.x it would save me a lot of hassle each time.

Clearly io_uring wasn't super compelling initially, but that's only because we had many more prevalent performance issues to address; now that things are getting really smooth within our libraries, the overhead of those syscalls is getting too nasty - we'll need this :)

[frank-dspeed]

LGTM +1 saw this commit and i am Frank1981 👯‍♂️

[vietj]

@franz1981 where are we at regarding this PR ?

[franz1981]

Hi @vietj Here https://github.com/eclipse-vertx/vert.x/pull/4164#issuecomment-1141100555

I am just come back from the summer holidays - will be back on my desk on Monday 👍