Different IO Operations

[MagicRB]

I'd be interesting in support for other IOPs, specifically socket recv/send with SOCK_STREAM and the new (from 6.14I think) FUSE operations. I'll eventually try to fit this into blockio-uring myself probably, but I want to ask if such a change would be accepted, where precisely should I start and if there is something blocking it in the current architecture of this library.

[jorisdral]

We welcome any contributions! I don't immediately see a problem with supporting recv/send, but I also wouldn't be able to say for certain that there aren't problems we'll run into. @dcoutts might know more. I'm not familiar with FUSE, so I'm not sure how that affects blockio-uring.

Roughly, the design of blockio-uring is this:

• a user submits a batch of IOOps using submitIO
• submitIO prepares SQEs for the IO operations and submits them to the io_uring context
• submitIO tells thecompletion thread about these SQEs
• the thread performing submitIO waits until it hears back from the completion thread
• The completion threads collects CQEs and sends IO result back to the submitIO thread if all its CQEs are collected
• The submitIO thread receives IO results and returns them to the user

The io_uring context and completion thread are both set up when calling initIOCtx. In practice, there is an io_uring context and completion thread for each capability in the GHC run-time.

If you want to get started with adding socket operations, my suggestion would be start with adding a new constructor to IOOp and see where the compiler complains. After that, the important bit is to figure out what the unhappy paths are when dealing with sockets. Is there anything that will disrupt the design I mentioned above? What happens if a socket is closed? What happens if there aren't enough bytes to read from the socket? Will anything block? Also, as discussed in #43, pinnedness and liveness of buffers is important, but we guarantee this already for IOOpWrite and IOOpRead so you can probably just copy the approach there

[dcoutts]

The main thing to keep in mind is that the current API is designed for batches of operations in a thread-synchronous style.

If what you really want is single operations in individual Haskell threads, then one would probably want a different high level API (but the underlying things could be shared).

But yeah, as Joris says, in principle that IOOP type could be extended with more operations.

[dcoutts]

Ah here's one issue:

if you're doing operations on sockets then those can block, and so it becomes reasonable to have 1000s of such blocked operations outstanding at once. This aspect would be incompatible with the current API & impl which assumes only non-blocking operations and so imposes a concurrency limit. For non-blocking operations you'd want no concurrency limit at all. And note that we've encountered uring bugs to do with overflowing the completion ring that we have so far not fully understood and have just worked around by ensuring the completion ring is big enough.