IO_Uring send: Broken pipe

[aubi1kenobi]

Hi guys,

Here is the error:

2023/05/02 15:55:23|ERROR|th=00007F95DBA21B00|/x.cpp:388|sendMessage:SEND FAILURE: errno=32(Broken pipe)

• 2 separates threads trying to send to the same end point
• Both connects, 1st thread sends and receives msgs no probs.
• as soon as 2nd thread sends a msg, 1st thread gets the above msg when trying to send

[beef9999]

There should be no problems, according to your description. Could you show me your code?

[beef9999]

And the server you are using is photon as well?

[aubi1kenobi]

I don't know what they use for server app, but most probably not. That shouldn't make a difference, should it?

The code is doing nothing else but create two threads, and have the 2 threads connect and send msgs to an ep.

The broken pipe error is really api based. I ddk the ins and outs, will have to dig deep, but it seems two clients cannot send and receivemsgs to the same ep. Idk how the underlying iouring implementation for this use case is.

You don't expose any underlying things, so it's difficult to try different settings.

I would like to try a socket pool, but don't how yet. I did ask, but got no response. I tried passing a connection reference from one thread to the other, but thats also not working. still investigating.

[beef9999]

It has been through a holiday. Will give you feed back about the connection pool later.

As to the previous broken pipe issue, I think you need to verify your server is working properly, and then show some client code, it will be helpful.

[aubi1kenobi]

Thanks. The server is a big company with 100s of thousands clients connecting, and maybe 100s of servers to service their clients. There is no experimentation in this line of business, its commercial, and driven by latency and proven hpc tech.

[aubi1kenobi]

I will send you the related lines of code. In the meantime, last night I kept thinking about:

• the 'iouring_tcp_client' must create a socket underneath, but none is provided to the user
• Same with the 'ISocketStream' (connection). although it has a 'get_fd', it doesn't have a 'set_fd'.

Also: the http server/client model, which is used by most people in the public domain, and probably most of your users, is a completely different model, with different goals, and some goals are even mutually exclusive to ours (think 'old', pure client/server model):

• No files to manipulate (read/write/send/recv, etc. None)
• It's all about 'msg buffers' sent/received on a tcp socket
• It's all about latency (efficiency of processing those buffers and connection), not throughput (msg/second). So 'fast' means something different to us.
• Therefore, 'connection' settings are/should be different:

• There is no 'One size fits all', which you do by 'hiding' and, therefore, deciding for us what the iouring and socket settings are

Some concepts are the same, but we only think 'client'. The server, as mentioned above is 200% working and independent from us (the custumers, 'clients').

[lihuiba]

Hi guys,

Here is the error:

2023/05/02 15:55:23|ERROR|th=00007F95DBA21B00|/x.cpp:388|sendMessage:SEND FAILURE: errno=32(Broken pipe)

• 2 separates threads trying to send to the same end point
• Both connects, 1st thread sends and receives msgs no probs.
• as soon as 2nd thread sends a msg, 1st thread gets the above msg when trying to send

I suppose x.cpp is your app code.

Does each thread have a separate connection? Created by new_tcp_socket_client() or new_iouring_tcp_client()?