[pdpix] `demi_pushto()` for TCP Does Not Work with Several LibOSes

[anandbonde]

Description

demi_pushto() for TCP does not work with Catnip, Catcollar and Catpowder.

How to Reproduce

Steps to reproduce the behavior:

1. Use demi_pushto() on a TCP demi_socket.
2. Build demikernel with any one of [Catnip|Catcollar|Catpowder].
3. See error.

Expected Behavior

There should be no error. TCP communication should work fine.

Alternative Solution

1. Check if the peer IP is what we have already connected to.
2. Return Not Supported for TCP sockets.

[BrianZill]

Interesting, I was about to say I didn't think this was a bug, because I wouldn't expect demi_pushto() to work with a connected mode ~socket~ queue descriptor. demi_pushto() is intended to be used with connectionless protocols (e.g. UDP), not TCP. So I would think specifying an address would fail for a connected queue descriptor.

But when I went to look up what error code Posix's "sendto()" function returns in this case, I discovered it doesn't return an error. Instead, it just silently ignores the provided address and sends the data to the connected peer. I'm not sure that's a great behavior either. I don't recall that being how BSD originally operated, but I may be wrong. Linux does return an error (EISCONN), but only if an address is provided (it treats the parameter as optional, with a NULL pointer meaning not provided).

So, I think before we "fix" this, we should decide what we want the behavior to be. @anandbonde, what do you mean by "TCP communication should work fine" -- that it should ignore the provided address?