The road to zig std

[ikskuh]

There are plans to upstream this library into the zig std library. This issue is here to create a concrete plan and track progress.

Some considerations:

• Define a clear public API/error surface
  • Collapse unix/windows/… errors into well-defined zig-network errors
• API must be event loop compatible
• Changes in zig std to OS APIs (https://github.com/ziglang/zig/issues/6600)
• Support unix sockets? If yes, how to solve it on windows
• Should we keep the address/port separation? OS's don't do it, but I find it unintuitive
  • How does this play with unix sockets? Address is a file name, not an IP + Port
  • The separation is required to implement happy eyeballs nicely
• Optimize the polling API (make it allocation-free if possible)

Roadmap:

• [ ] Clear up remaining questions
• [ ] Clean up the API and provide abtractions
• [ ] Create dedicated fork and start integration
• [ ] Create PR and hope for the best

ping @alexnask @Luukdegram @kristoff-it

[alexnask]

API must be event loop compatible

This is already the case, if I'm not mistaken (well, not completely on windows, I believe interacting with AFD is necessary for async connect, bind etc.)

Support unix sockets? If yes, how to solve it on windows

I am not familiar with unix sockets tbh, I will look at examples and differences with other sockets and come back to this issue.

Optimize the polling API (make it allocation-free if possible)

On windows, I intentionally made the socket set reallocate to support an arbitrary number of sockets.
In WINAPI, they actually provide a fixed set with a size configured through a macro, maybe we can expose socket sets generic over a user-provided size.

[Luukdegram]

I feel the API should be flexible enough to enable integration with Unix sockets at one point. I believe, however, that Unix sockets are out of scope as of now as I see it as a separate thing (Only current device, rather than over the network like TCP/UDP). It's easier to get things merged if the steps we take are small.

Personally, I don't mind the polling API to have allocations as I can imagine the user not always knowing how many it will open (at comptime). I do realize, however, that I come from a desktop/server view, and haven't given it much thought from an embedded viewpoint. Perhaps we could make the API flexible enough to allow for both? I thought of making the polling function accept a slice, but that would require the user to create the sockets themselves which is a pity. Probably what Alex suggested sounds pretty doable.

I think the best way to get this into std, is by making a PR sooner rather than later and get feedback that way. I'm sure a lot of people will have opinions to share on how to go further. (Especially since they most definitely have more experience than me).

[kristoff-it]

Yo, I just want to point out that there are a few people operating in this space at the moment and that I think making API decisions is a non-trivial problem that requires to coordinate & communicate effectively.

You already correctly identified LemonBoy's effort, on top of that there's also @lithdew who is working on a bunch of async APIs for windows, both network and filesystem IIRC, and he also plans to upstream some code. Please pay attention to his effort too, there's an ongoing discussion in #windows-evented-io (in the showtime discord).

Unfortunately at the moment I can't provide better support or advice than this, as I'm focusing on the website + showtime, but I'm happy to help with specific issues with a well-defined scope, just ping me in case.

[alexnask]

Yes, I will be looking into pike's AFD code for sure, seems interesting.
It also will determine some of the ways the event loop implementation has to change, at least for windows

[OfekShochat]

Will this be achieved? zig-network could be amazing in stdlib...