UDP progress

[kylegoetz]

I didn't want to edit someone else's issue, so I've created this one to track my thoughts/progress on UDP builtins (plus related additions to base)

The existing (TCP) Socket (et al.) currently has the following functionality:

• Socket
• ListeningSocket
• BoundServerSocket
• UnboundServerSocket (seemingly not used anywhere in base)
• Socket.accept : ListeningServerSocket ->{IO, Exception} Socket (with underlying impl
• Socket.client : HostName -> Port ->{IO, Exception} Socket
• Socket.close : Socket ->{IO, Exception} ()
• Socket.listen : BoundServerSocket ->{IO, Exception} ListeningServerSocket
• Socket.port : Socket ->{IO, Exception} Nat
• Socket.receive : Socket ->{IO, Exception} Bytes
• Socket.receiveAtMost : Socket ->Nat ->{IO, Exception} Bytes
• Socket.send : Socket -> Bytes ->{IO, Exception} ()
• `Socket.server : Optional HostName -> Port ->{IO, Exception} BoundServerSocket
• Socket.toText : Socket -> Text

"Simple" stuff

1. UnboundServerSocket has apparently no dependents in base
2. Socket.toText just stringifies Socket
3. Socket.port retrieves Port from Socket (Socket wraps HostName and Port in the Haskell)
4. Socket.receive is just receiveAtMost with a fixed number of bytes.
5. There are some unsafe raw functions in there that are thin wrappers around the built-in, and "cooked" versions that are safe. Users are encouraged not to use the raw versions.

Use cases

Creating a client

Socket.client yields a Socket that is connected to a given (HostName, Port). From there, a client can send and receive[AtMost] as operations on Socket.

Creating a server

Socket.server yields a BoundServerSocket, not yet listening but it has acquired a socket resource. From there, one listens, which yields a ListeningServerSocket and it listens for incoming connections. From there, one accepts an incoming connection (the function blocks until one arrives), which converts the ListeningServerSocket into a Socket. At this point, like with a client, the server can send and receive.

Closing

Socket can be closed when no longer in use.

UDP

The leading UDP library for Haskell seems to be Network.UDP. It largely works the same as TCP.Simple (which is used for the existing Socket functionality). However, there are a couple differences I'll highlight, and I'll use the Haskell sigs instead of potential Unison ones here:

-UDPSocket (equivalent to Socket)

• ListenSocket (equivalent to ListeningSocket)
• ClientSockAddr (no analogue in TCP, but used to represent the Socket address of a connected (or attempting-to-connect) client
• No equivalent for BoundServerSocket
• No equivalent for UnboundServerSocket
• accept : ListenSocket -> ClientSockAddr -> IO UDPSocket (equiv. to Socket.accept : ListeningServerSocket ->{IO, Exception} Socketbut requires theClientSockAddr` to be known)
• clientSocket :: HostName -> ServiceName -> Bool -> IO UDPSocket (Socket.client : HostName -> Port ->{IO, Exception} Socket; ServiceNamewrapsPort, Boolparam connects to the host iftrue`)
• close :: UDPSocket -> IO () and stop :: ListenSocket -> IO () (Socket.close : Socket ->{IO, Exception} ())
• No equivalent to Socket.listen : BoundServerSocket ->{IO, Exception} ListeningServerSocket since the UDP library initializes the server socket in listening state
• No equivalent to Socket.port : Socket ->{IO, Exception} Nat
• recv :: UDPSocket -> IO ByteStream (equivalent to Socket.receive : Socket ->{IO, Exception} Bytes)
• recvFrom :: ListenSocket -> IO (ByteStream, ClientSockAddr) (no equiv in base; this lets you receive from a ListenSocket that hasn't accept`ed a connection and will give you the data plus info about the sending client)
• No equivalent to Socket.receiveAtMost : Socket ->Nat ->{IO, Exception} Bytes
• send :: UDPSocket -> (ByteStream -> IO ()) (equiv to Socket.send : Socket -> Bytes ->{IO, Exception} ()`)
• sendTo :: UDPSocket -> ByteStream 0> ClientSockAddr -> IO () (no equivalent, similar to UDP.send, analogous in purpose to UDP.recvFrom)
• serverSocket :: (IP, PortNumber) -> IO ListenSocket (equiv to `Socket.server : Optional HostName -> Port ->{IO, Exception} BoundServerSocket except it skips Bound and proceeds right to Listen state)
• NO equivalent to Socket.toText : Socket -> Text

Ideas

If we want to keep UDP as close to TCP as possible for UX (similar terms, types), we can do this, but eschew the Unbound and Bound socket states/types by building in:

• [x] ##UDPSocket
• [x] ##ListenSocket
• [x] ##ClientSockAddr (maybe?)
• [ ] ##IO.UDP.ListenSocket.accept.impl.v1 : ListenSocket -> ClientSockAddr ->{IO} Either Failure UDPSocket
• [x] ##IO.UDP.clientSocket.impl.v1 : HostName -> Port -> {IO} Either Failure UDPSocket
• [x] ##IO.UDP.UDPSocket.close.impl.v1 : UDPSocket ->{IO} EIther Failure ()
• [x] ##IO.UDP.ListenSocket.close.impl.v1 : ListenSocket ->{IO} EIther Failure ()
• [x] ##IO.UDP.recv.impl.v1 : UDPSocket ->{IO} EIther Failure Bytes
• [x] ##IO.UDP.recvFrom.impl.v1 : ListenSocket ->{IO} EIther Failure (Bytes, ClientSockAddr)
• [x] `##IO.UDP.send.impl.v1" : UDPSocket -> Bytes ->{IO} EIther Failure ()
• [x] ##IO.UDP.serverSocket.impl.v1: Optional Text -> Text ->{IO} Either Failure ListenSocket (non-impl will be Optional HostName -> Port -> ...)
• [x] ##IO.UDP.UDPSocket.toText.impl.v1 : UDPSocket -> Text
• [x] ##IO.UDP.UDPSocket.port.impl.v1 : UDPSocket -> Nat (non-impl should cast to Port instead of Nat)
• [x] ##IO.UDP.sendTo.impl.v2 : ListenSocket -> Bytes -> (Text, Text) ->{IO} Either Failure ()

I'm not sure about the utility of sendTo and send and accept. I need to read more about how UDP sockets work and do testing. I find it strange that a server would need to accept a ClientSockAddr connection, but require you to already know it before accepting, but I don't see a function to detect an attempted connection.

More testing needed.

current state https://github.com/unisonweb/unison/commit/354ced3ef8e4be18ea0235d8b225d3b5b01396cd

[aryairani]

Awesome, thanks for tracking here

[kylegoetz]

After struggling mightily with why I couldn't get the UDP library's accept to destructure the Socket from ListenSocket work by hooking into Network.Socket.accept (the built-in UDP lib's accept has as one of its args a given ClientSockAddr which is an impossibility for listening for any client connection to accept, I discovered this talking about the same error I was getting in the Haskell:

https://stackoverflow.com/questions/36325912/socket-error-errno-102-operation-not-supported-on-socket

You are using a udp socket, SOCK_DGRAM, and udp does not listen for connections, it receives each message on its own Use recvfrom to receive udp messages

recvFrom :: ListenSocket -> IO (ByteString, ClientSockAddr) works, and I suppose then accept :: ListenSocket -> Socket only exists to prepare a UDP server to also be a client that can send via sendTo :: ListenSocket -> Bytes -> ClientSockAddr -> IO ().

So now I'm leaning toward ClientSockAddr being exposed to Unison, but nothing to modify it. Maybe a toText, not no way to construct it since it seems for UDP purposes, its only relevance is to re-feed it into a function like sendTo so a server can send to a client after a client has already send data (??).

[aryairani]

@kylegoetz This is awesome. Paging @dolio to help w/ any runtime questions, and @runarorama to help w/ any design questions. We can create a Discord thread as well.

[runarorama]

Yeah it looks like the only way to get a ClientSockAddr is via recvFrom. It extracts the peer address from the datagram. So I think you're right that accept prepares a UDPSocket so the server can respond. Seems like this library is trying to provide a TCP-like API so that e.g. you don't have to specify the address each time and buffers can be reused by hanging them on the simulated "connection".

[kylegoetz]

Yeah given that accept doesn't have good documentation, is seemingly just sugar, and when I use it, my Haskell hangs at accept without advancing to the next line of code, I think it's not worth including as a built-in.