NIO-compatible HttpPipeline

[John-Connolly]

I was wondering if there is an interest with integrating more with Swift-nio? Specifically allowing routes to return futures allowing users to use non-blocking db drivers. Looking through the Pointfree website code making this change would be pretty huge and probably have no tangible benefits for you guys as I suspect performance is not an issue. However would you guys be open to excepting a PR for something like and AsyncHTTPPipeline module? I have been experimenting with something like this and it ended up very similar to what is here.

[stephencelis]

This definitely sounds interesting! We haven't given much thought to what a deeper NIO integration may look like but are interested to explore. Right now we have compose connections using a mixture of an EitherIO for error handling, an IO for error recovery, and a Parallel for parallelism, though it's still driven by Dispatch. While this stack has been perfectly fine and flexible, we'd love to hone it by exploring something like ZIO. At that point, we may be able to have two interpreters, one that uses dispatch queues and one that uses NIO futures and an event loop.

I'm curious as to what your module look like? Something that swaps IO<Conn> out for Future<Conn>?

[John-Connolly]

Yeah that is essentially what I did. One of the things about NIO Eventloop futures is there is no way to have a pure function as you also need a reference to the event loop to construct a future. So I think the Conn would need to be changed a lot.
I've been experimenting with the idea here https://github.com/John-Connolly/concorde/blob/master/Sources/concorde/Http/Conn.swift

The ThreadCache uses NIO's ThreadSpecific variable to hold things like db connections for that event loop.

public final class Conn {

    let cache: ThreadCache
    let eventLoop: EventLoop
    let request: Request
    var response: Response

    init(cache: ThreadCache,
         eventLoop: EventLoop,
         request: Request,
         response: Response) {
        self.cache = cache
        self.eventLoop = eventLoop
        self.request = request
        self.response = response
    }

    public func future<T>(_ t: T) -> Future<T> {
        return eventLoop.newSucceededFuture(result: t)
    }

    public func promise<T>() -> Promise<T> {
        return eventLoop.newPromise()
    }

[stephencelis]

Yeah that is essentially what I did. One of the things about NIO Eventloop futures is there is no way to have a pure function as you also need a reference to the event loop to construct a future.

I wonder if for this reason, a design like ZIO would work well for NIO support/integration, since the interpreter could pass the event loop along. That very well may not work, though. I'd need to spend some time sketching it out.

[stephencelis]

We'd love to revisit all this stuff soon with the coming release of concurrency features in Swift. I'm going to convert to a discussion to capture things there. Thanks again for starting it (checks watch) almost 3 years ago 😄