Fix make-network-process to work with emacs 25.1+

[ebellani]

See https://www.reddit.com/r/emacs/comments/5azqjr/emacs_web_server_problems/d9mltkw For an explanation

[DamienCassou]

What is the status of this PR?

[jcaw]

@DamienCassou

Posted this on a few issues, quoting here for convenience:

The MELPA Recipe has been updated in leui of inactivity: melpa/melpa#6184

Active fork here: https://github.com/jcaw/elnode

The fork is functional on Emacs 26+, should work on Emacs 25 too. Haven't had a chance to test on 25 yet.

[nicferrier]

This is odd.

I haven't seen Emacs > 24 yet but nil in the nowait parameter would suggest that it's not doing async connect anymore?

[jcaw]

Yeah it's super bizarre, doesn't seem to be documented anywhere. It took web-server out for a while too, among other packages.

In 26, they increased the asynchronicity of the networking code.

** The networking code has been reworked so that it's more
asynchronous than it was (when specifying :nowait t in
'make-network-process'). How asynchronous it is varies based on the
capabilities of the system, but on a typical GNU/Linux system the DNS
resolution, the connection, and (for TLS streams) the TLS negotiation
are all done without blocking the main Emacs thread. To get
asynchronous TLS, the TLS boot parameters have to be passed in (see
the manual for details).

Certain process oriented functions (like 'process-datagram-address')
will block until socket setup has been performed. The recommended way
to deal with asynchronous sockets is to avoid interacting with them
until they have changed status to "run". This is most easily done
from a process sentinel.

Annoyingly there's no meaningful detail about servers.

My best guess is that this meant make-network-process was returning before the server was set up, which caused cryptic errors and silent failures, so they explicitly disabled that behaviour. But I'm not privy to how it works at a low level. Do you know how servers were actually handling :nowait? How much of each connection was happening outside the main thread?

This seems to be the most complete discussion of the issue on GitHub. There's a core development discussion here.

Another discussion from end-users here, nothing new though. The web-server issue I linked earlier notes the behaviour was still present in 27dev, as of September last year.

Annoyingly, some people also seem to have the problem on 25, but it seems rare.

It looks like most packages are using a conditional :nowait flag now. They flip it off on 26 upwards.

I found one resource early on in debugging this that said they changed the way it worked in Emacs 25 so the :nowait parameter was silently ignored for servers. In 26.1, the change was formalised and it throws an error. No idea where I saw that, though. I can't track it down.