kv-cache : refactor + add llama_memory_state_i#13746

Merged

ggerganov merged 14 commits intomasterfrom

gg/kv-cache-simplify-part3

May 31, 2025

Commits on May 30, 2025

Commits on May 31, 2025

server : update batching logic to reset n_batch on successful decode
ggerganov
committed
server : upon full re-processing, remove the sequence from the cache
ggerganov
committed
kv-cache : add TODO for doing split_equal when split_simple fails
ggerganov
committed