Native channels on JRuby don't work with JDK builtin selectors

[headius]

There are a number of IO types in JRuby that don't come from the standard set of channels on the JDK. Primary example of this is UNIXSocket/Server, which are provided by jnr-unixsocket using jnr-enxio. jnr-enxio, because it uses FFI to perform all IO and selection operations, has its own provider and its own selectors.

In nio4r, selectors are always created using the default provider, so they will only work with channel types from that provider.

As a result, nio4r selectors do not work with JRuby's UNIXServer/Socket, and this is the root cause of jruby/jruby#2750.

The issue is further complicated by the use of IO.pipe channels to wake up the selector. IO.pipe channels in JRuby come from the default provider, and you can't use a single selector to handle channels from two different providers.

In JRuby, we mitigated this issue in our selector logic as follows:

• If we're only selecting on one channel or multiple channels with the same provider, just get an appropriate selector for it from a pool. The pool is keyed off provider.
• If we are selecting against channels from different providers, the enxio selection happens in a separate selector and thread with a pipe connecting it to the default provider and selector; this allows either selector to wake both.

It's not an ideal setup, but I think nio4r will need to duplicate this logic, or we need to expose the logic we have in JRuby more directly (i.e. not hidden behind IRubyObject wrangling).

[headius]

Responding to @tarcieri comment on jruby/jruby#2750:

A standard API would make sense. There's an additional problem here, though, in that the selector logic lives within a single instance of NIO::Selector, which only aggregates one selector. Perhaps it should aggregate a selector instance for each provider it encounters, and then hit the logic I mention above?

We can certainly pull out the selection logic for multiple providers as a standard API, but I'm trying to figure out what that should look like to support NIO::Selector.

[tarcieri]

@headius what you're describing makes sense, but how do you multiplex across multiple selectors?

[headius]

Well the unfortunate way it works now is like this:

1. Main selector also selects against a pipe.
2. Secondary selector starts a thread waiting on select.
3. If the secondary selector fires, it writes a byte to the pipe to wake up the main selector.
4. Main selector sees read operation on the pipe and digs out the secondary selector's key+channel.

That's obviously not great, but it can be improved by only using one selector if all channels are homogeneous (same provider) or by offloading the additional selector thread to a secondary reactor.

[tarcieri]

Failing everything else, I think these sort of hierarchical selector setups can be written on top of nio4r, as opposed to requiring first-class support. First-class support would be nice though.

Probably isn't going to happen any time soon though, as #95 is eating up all the time I have to spare on this gem.

[headius]

I want to make clear this only affects UNIX sockets; all other socket types currently are standard JVM channels, for which the base selector will work fine.

Maybe we can improve our UNIX socket impl.

[ioquatix]

@headius is this still an issue?

[headius]

@ioquatix Never saw your comment, but yes this is still a problem. The line in question is here:

https://github.com/socketry/nio4r/blob/master/ext/nio4r/org/nio4r/Selector.java#L56

This will create a JDK Selector using the default (built-in) provider, which only works with the standard socket types on the JDK. In other words, it will not work with JRuby's custom UNIXSocket classes.

In order to get a selector appropriate for a given nio SelectableChannel, (which would cover all selectable channels in JRuby, including TCP/UDP sockets and our UNIX socket impl) you can use channnel.getProvider().openSelector(). The structure of nio4r makes that a little trickier, though, because it creates a Selector long before you ever register any IO objects with it.