How to avoid green color before the command is really complete with autocompletion active

[cstoyanov-cs]

$ ble summary

GNU bash, version 5.3.3(1)-release (x86_64-pc-linux-gnu) [Arch Linux]
ble.sh, version 0.4.0-devel4+8060b7ad (noarch) [git 2.51.1.dirty, GNU Make 4.4.1, GNU Awk 5.3.2, API 4.0, PMA Avon 8-g1, (GNU MPFR 4.2.2, GNU MP 6.3.0)]
bash-completion, version 2.16.0 (hash:480ffcc6a751e55621ec526eb5dea7a0d86d9e72, 17877 bytes) (noarch)
locale: LANG=fr_FR.UTF-8
terminal: TERM=xterm-kitty wcwidth=16.0-west/16.0-2+ri, kitty:0 (1;4000;43)
options: +extglob +histappend -hostcomplete +inherit_errexit

Hello, first of all thank you very much for your work on the script. I have a question, im not sure if its a problem or a feature request but when I activate autocompletion with bleopt complete_auto_complete=1. Lets say in the example of clear command, I type cl and the auto completion suggest the word clear for me. But even if I don't accept the suggestion my cl uncomplete and not valid command turn to green.

Is there a way to keep the actual command we are typing red if its not yet a valid command. It is the case if I deactivate autocompletion with : autocompletion with bleopt complete_auto_complete=
But I like autocompletion, I just would like if color could work like fish for example. In fish, the command clea is red even if the autocompletion suggested the r. if I don't validate it its stay red. Im not saying that to compare both I know blesh is not fish. I just think it can help to understand the problem.
Actually sometimes I press enter because my command is green but I forgot to accept the autocompletion so the command isn't valid.

I tried this config but it doesn't work :

ble-face syntax_error='fg=red,bold'

ble-face command_suffix_new='fg=red'

ble-face syntax_command='fg=blue,bold'

ble-face command_file='fg=blue'

ble-face auto_complete='fg=242'

Thank you for your help and again thank you for your work.

[akinomyoga]

This was discussed in #5 (comment) just after it was first implemented. This is the machine translation of the comment and replies to it:

---

Translation of #5 (comment) by @cmplstofB

I'd like to talk about a somewhat subtle issue regarding rendering during auto-completion.

For clarity, let's assume the following: the ls(1) command exists, and there is no command named l (nor any alias set for l).

After executing the command line

$ ls /tmp

suppose we are now in the state:

$ l@

(here @ indicates the cursor position).

In this state, the string l is rendered as it would be after auto-completion---in this example, auto-completion would produce ls /tmp, so the l is drawn according to the style defined for command_file.

However, in reality, since there is no command named l, running this line as-is would result in an error. If we prioritize correctness of the result, shouldn't the rendering of l in this state follow the style for syntax_error instead?

...That said, I'm not entirely sure myself. One could also argue that "auto-completion can be regarded as a view of the completed command line, so when rendering part of it, the reference should be the fully completed command line."

@akinomyoga, do you use the latter interpretation when implementing the current rendering behavior? Or is there another reason for this (for example, implementation difficulty or performance concerns)?

Translation of #5 (comment) by @akinomyoga

Actually, I've been a bit conflicted about this myself. At first, I thought it would be more natural to apply syntax highlighting excluding the auto-completion part. However---as you pointed out,

Or is there another reason for this (for example, implementation difficulty or performance concerns)?

---the main reason is that implementing it properly would be quite troublesome.

For reference, when I looked at how fish handles this:

• When auto-completing a command name, it colors it based on whether the command exists before completion.
• When auto-completing a file name, it seems to underline it based on whether the file exists after completion (or more precisely, whether the current input is a prefix of an existing file name).

So, since either interpretation seemed reasonable, I decided to just leave it as-is.

As for the auto-completion mechanism itself---it's actually implemented by inserting the completion string directly into the buffer, so figuring out a clean way to handle this isn't straightforward…

Here are some of the approaches I've considered:

• (a) Let the parser recognize the "temporarily inserted string" and skip over it. The parser is designed to update incrementally (only the parts that changed since the last parse), so we'd have to redesign how to represent the diff information that includes "temporarily inserted strings." And on reflection, it feels wrong from a design perspective to add such an unrelated ad-hoc feature into the parser.

• (b) Implement it without actually inserting the string. But then, when calculating character layout on screen, we'd have to account for "objects that are displayed but not part of the actual edit buffer," which also sounds complicated.

• (c) Another idea I considered before implementing was to display it using a separate overlay window. That would require setting up a simple window system, but the implementation might stay relatively clean. (Note: It would still be complicated, and since I haven't fully examined the details, there could be unexpected difficulties later.)

  When I looked into this before, I found the following pages describing four types of preedit input methods: "over-the-spot," "on-the-spot," "off-the-spot," and "root-window." The current implementation corresponds to on-the-spot, while using a separate window would be over-the-spot (or possibly root-window).

  • Big Sky :: On Vim and Japanese Input (in Japanese)
  • Seamonkey Input Method Specification

Addendum: If we continue with the on-the-spot approach, option (b) might be preferable---but after checking the source code, it still looks quite tricky. Like the parser, the layout calculation is also designed to update only the changed parts, so we'd have to track two kinds of diffs: 1) diffs for the actual edit buffer, and 2) diffs including the temporary preedit content. Hmm… it'd be nice if there were a better approach…

Translation of #5 (comment) by @cmplstofB

I thought so as well…

I actually didn't know about fish(1)'s behavior (or rather, I posted without even checking it), but the fact that it's implemented that way probably means that this issue isn't widely recognized---or that there's no clear consensus among developers---or simply that it's quite troublesome to implement properly.

As a layperson, I had just assumed that ble.sh and other shells implemented auto-completion in a way similar to your option (b). Apparently, that's not the case.

Incidentally, I've also read the web pages you mentioned (though that's neither here nor there). I actually came across the terms over-the-spot and on-the-spot when I was struggling to get Japanese input working correctly in the rxvt-unicode-256colors terminal emulator---I kept tweaking settings and experimenting with different configurations. In the end, I gave up and switched to a terminal emulator based on LibVTE, but I never thought those same concepts would come up again in the context of implementing auto-completion!

Translation of #5 (comment) by @akinomyoga

As a layperson, I had just assumed that ble.sh and other shells implemented auto-completion in a way similar to your option (b). Apparently, that's not the case.

I think that's probably specific to ble.sh, since it's implemented in shell script---other shells are likely different. When something is implemented in shell script, there are inevitably limitations on the data structures you can use (it's possible to represent complex ones, but it's extremely cumbersome and very slow).
So in short, it comes down to data-structure constraints and performance.

(For anyone interested:) A proper text editor typically keeps the editing buffer line-oriented (or stores each line's range using a separate data structure). That way, character layout calculations are localized within each line. Since such editors are implemented in fast languages, they can afford to recalculate the entire line without performance issues. With a well-designed data structure, supporting something like the “inserted object” in option (b) can be done cleanly.

In contrast, in ble.sh, due to speed concerns, the entire editing buffer is stored as a single string. Even for one-line cases, re-parsing and recalculating the layout from scratch each time would be unbearably slow, so it only updates the minimal region that actually changed. Because the data structures are quite primitive (I wish I could just have an array of structs...), adding support for something like “inserted objects” would require modifying many parts of the code, and it would have a direct impact on performance.

For comparison, zsh-syntax-highlighting only handles syntax coloring (the layout calculations are done by zsh itself), and it doesn't perform full parsing---it only applies simple highlighting. Even so, it recalculates the entire buffer each time, so with long command lines, there's noticeable input lag (at least when I tested it a few years ago).

For now, I think I'll leave this issue as-is until a good implementation idea comes up, and won't try to address it immediately.

---

In summary, the primary reason for the current behavior is that it would be extremely tedious to implement the suggested behavior. Currently, the syntax highlighting (the parser) and auto-complete are implemented as independent modules, and their implementations are orthogonal to each other. However, to implement the suggested behavior, the parser (which is already complex) needs to additionally take account of what auto-complete is doing, or we need to re-implement the entire infrastructure on which both features are based.