llama.vim : cache results even if cursor moved

[ggerganov]

This change improves the cache utilization. Before, we discarded FIM results which were not displayed (for example, because of the cursor moving or the user entering new text). With this change, all FIM results would be cached, regardless if there have been user interactions since the last request.

For example, in the following case, typing he quickly will now reuse the cache result from the h input and the client will send only 1 request to the server (in contrast to sending 2 requests on master):

• master:

• PR:

[ggerganov]

Been using this feature today and it works really well. The cache mechanism is really good. When the multiple-generation feature is ready (ggerganov/llama.cpp#11142) we can even cache multiple completions for even better cache utilization.

[VJHack]

This is really nice optimization and caching seems to work really well!

Just one thing, I'm experiencing this strange behavior where when I move the cursor to the left in insert mode on a cached completion, it keeps my suggestion displayed inbetween the text I typed. Are you experiencing this as well?

print("Calculating is cached and I moved my cursor to the left a few times. The suggestion doesn't disappear. Let me know if you can reproduce.

[ggerganov]

Yes, I can reproduce it. It's behaving like this also on master, so it's not caused by the changes in this PR.

[ggerganov]

Should be fixed now: cf684b4

[VJHack]

Nice! Works very well and thanks for taking care of the caching bug.