[Discussion] Would you evolve the project to use zig's new async I/O?

[gjuchault]

Hello !

Thank you for all the work you've achieved in the Zig ecosystem 😄

Are you any hyped by the zig announcements and demos about the upcoming async I/O upcoming in 0.16?
Would you switch http.zig from a threadpool to async I/O?

Sort of related to #24

Thanks!

Edit
I see on the mentionned issue that you'd probably wait for the std/common dependencies to http servers like pg and others to be async as well; I would say that in the Node ecosystem, node is fully async I/O, but all the PG clients I'm aware are using a connection pool (cost of creating a connection, limits on PG, security, etc.), and the native FS module is using a threadpool

[karlseguin]

I haven't been following too closely, but yes, the goal is to align with Zig's I/O at some point. Whether that's 0.16 or not depends on what exactly ships in 0.16. If the full green thread implementation works on Linux/Mac/Windows, then yes, I'll probably switch (maybe it already does? but last I saw, it was just Linux right now).

If the only fully working implementation is the thread pool...I don't know.

[zigster64]

My 2c opinion - yes, it would be nice to use stdlib green threads or thread pools as handler functions after 0.16 lands - with the user supplying an Io implementation to the http server.

Especially for long lived handlers that stream (which is a big use case for me personally). I’m fine for now using short lived handlers, and a long lived array of subscriber streams to publish to. Works great, but being able to use green threads would be a nice alternative too.

0.16-dev is still a fast moving target, and doesn’t support kqueue just yet (last I looked) … which sort of limits experimenting with this just yet. Being on Mac for dev and bsd for prod … kinda have to wait

[zigster64]

Might be able to experiment soon, with appropriate caution

https://ziglang.org/documentation/master/std/#src/std/Io/Kqueue.zig

No idea if it works, but it does have both x86 and arm64 asm for context switches, and something called a “Fiber” floating around in the code as well

If it works, will start playing with the concepts using stdlib http server first, and see how it all fits together

[zigster64]

Shower thoughts on where to add the new IO param :

• Want to make any changes minimal for existing code using http.zig
• Want to make it flexible enough so the user can mix and match IO strategies per route if needed

I think the handlers will need to be able to access the Io implementation - but its probably best to add the Io to the res struct that already gets passed in to each handler (alongside arena, etc)

I think adding an Io value to the Server.Config struct will be simple enough to allow the user to define the "default IO implementation" to be used by both the worker pool, and across all the handlers.

Then adding an optional IO value to the Router.RouteConfig struct will enable users to override the IO implementation for a given route. Use case for this - might want to use ThreadPool as the default for all short lived handlers, and coroutines for long lived streaming responses so as not to hog all the threads.

No change needed to the Dispatcher function, as the IO value is stored against the res already, if it needs to use that.

In Server.handleRequest() - it creates the res object here, so stick the Server's default IO into the res at this point, then override it if the found route has a customised Io value in it's config.

If the user has a custom handle() func - no change, as the appropriate Io param is part of the res already.

So for the above - that handles propagating the correct Io value per route all the way through to the handler, with minimal changes to any user code.

---

Only tricky bit then is in worker.handleConnection(). It's already in a thread pool at this point. Presumably, if looking at replacing the httpz.ThreadPool with a user-specified IO implementation, then the whole worker code would just adopt the Server's default IO here and use io.concurrent() in its place or something. Not 100% sure how that works yet.

That means that calling handlers that use the server's default IO don't need anything special - because handleConnection() is already wrapped in the server's default IO concurrent() call, if that makes sense.

Just need to handle the tricky case where there may be a custom IO defined for a route. I guess the worker would just spawn a whole new route_specific_io.concurrent(handlerFn, args) or something, and then disown / detach that handler to look after itself from there on.

Server.handleRequest() assumes that the dispatch (or middleware) is called synchronously relative to the handleRequest() call, so it can catch any errors and report 500 on fail. I think this is still fine, because that call to Server.handleRequest() is already nicely wrapped by io.concurrent() for either the default case, or the route-specific case.

I think that's correct ... will have to see what it looks like in reality.

If true, it means not a whole lot of changes ... just a bit tricky to reason about at first.

[karlseguin]

I think there's two parts to this.
1 - Exposing an std.Io to handlers. I agree this seems pretty straightforward.
2 - Replacing the Blocking and Nonblocking workers and the threadpool with std.Io. This seems less straightforward, or maybe it's just the lack of documentation. Ideally, worker.zig is largely removed. You get a connection and server.concurrent(newConnFn, socket)

[zigster64]

Heads up - got a web server of sorts compiled and running at least with kqueue using std.Io.Evented on Mac. So it is possible at least.

A fair bit of extra ceremony needed, but I do like this new API.

Going to have to shelve the concept for a while, because 0.16-dev basically isn't anywhere near ready yet, especially on Mac. Of note :

• The underlying std.Io vtable has now grown a bit - changes implemented in Threaded, but not Evented. You can get it compiling by patching Kqueue.zig to add the missing functions to the vTable (like groupConcurrent etc, just set to undefined to get it passing the compiler).
• Too many functions in Kqueue.zig are not implemented yet. Basic things we need like listen() and accept(). Grep the code for @Panic("TODO") and you will see what I mean

What I might be able to do in the interim is build a web server using http.zig as it is + zig 0.16 ... and then manually use std.Io.Evented and io.concurrent() to custom roll some coroutine driven handlers. A pointless exercise in itself, but good for a throwaway experiment for now.

The new-new Io is actually very nice, and im looking forward to the day it's closer to working. But not today