[BUG] Sending selection value to REPL fails for very large selection

[pylasnier]

Version of Kakoune

v2024.05.18

Reproducer

When sending text selection to tmux by repl-send-text, if the selection is too large (~1k lines), it sends the last thing it sent rather than the current selection. Trying to use the %val{selection} expansion or derived forms e.g. %sh{echo $kak_selection} has the same issue, however the selection expansion does not have this issue without using repl-send-text, e.g. echo %val{selection} has the expected behaviour.

Outcome

It sends the last thing it sent rather than the current selection.

Expectations

repl-send-text should send the text selection to a REPL.

Additional information

NixOS unstable, run in tmux for REPL commands

[Screwtapello]

When the problem occurs, is there any error message in the *debug* buffer?

The tmux repl plugin implements repl-send-text by using the tmux set-buffer command to set the contents of a tmux buffer, and then tmux paste-buffer to make tmux type the buffer contents into the target window. It sounds like the set-buffer operation is failing for some reason; perhaps because the selection is too large to fit on the command-line.

If you have GNU xargs installed, you can check what the effective command-line limit is with echo | xargs --show-limits.

[pylasnier]

Yes that seems to be the issue! *debug* buffer after sending the large selection:

shell stderr: <<<
command too long
>>>

This is my echo | xargs --show-limits output:

Your environment variables take up 7567 bytes
POSIX upper limit on argument length (this system): 2087537
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2079970
Size of command buffer we are actually using: 131072
Maximum parallelism (--max-procs must be no greater): 2147483647

I suppose this is a limitation of the way kak interfaces with tmux, by shell? Maybe not a bug then, but I would be interested in some sort of REPL implementation with much larger selection limit.

For context, my use case is for an extension I am building for using kak with the interactive theorem prover HOL4. I'm using the REPL commands to send text selections to the HOL4 REPL. Theorems can be huge, and though not a deal-breaker, it would sometimes be nice to send a whole theorem to the REPL.

[pylasnier]

Interesting to note is that the command size limit does not impact using printf or echo, so my current solution is to use a mkfifo and pipe to it with printf, rather than using the repl which likely just does not suit my use case.

[Screwtapello]

Maximum length of command we could actually use: 2079970

When the tmux REPL code does something like:

tmux set-buffer "$kak_selection"

...then the text to be pasted is stored in an environment variable and then copied to the command-line, so you wind up with two copies. If there's 2,079,970 bytes of space available, then you can paste at most about a megabyte of data.

Other REPL implementations (X11, dtach) do something like:

printf '%s' "$kak_selection" | text-sending-command

Because printf is a shell builtin, the shell doesn't have to spawn an external command, which means it doesn't have to make a second copy of the text, which means it can handle twice as much text (about 2MB, on your machine).

I suppose this is a limitation of the way kak interfaces with tmux, by shell?

It could be changed to use the tmux load-buffer command, which works like the X11 and dtach tools, and would raise the limit (though not eliminate it entirely).

It would also be possible to make Kakoune write data directly to the text-sending-command that reads from stdin, although that would make the implementation much more complex.