User editible n_keep content

[toddwbucy]

I'm not sure if this is the right project for this idea, but am I the only one who wants the ability to manage the content of n_keep so that you can better manage context over long conversations and prevent that rabbit trail you went down three prompts ago doesn't continue to corrupt your model's output? A more detailed problem statement can be found below:

Problem Statement:
LLM users lack tools to manage their context window (n_keep) before hitting token limits, leading to disruptive conversation restarts.
Objective:
Develop a context management system enabling users to monitor, tag, edit, and optimize their context window content before submission to LLMs.
Scope:

Context utilization monitoring and alerts
Content tagging/editing interface
Version control and history tracking
Context visualization tools
Integration with existing LLM platforms

Expected Outcomes:

Reduced conversation disruptions
Improved context relevance through user management
Enhanced conversation flow
Increased user satisfaction through proactive context control

Success Metrics:

Reduction in forced conversation restarts
Context window utilization efficiency
User engagement with management tools
Maintained response coherence post-editing

This solution addresses the critical need for user control over context management in LLM interactions.

[dcssi]

One of the beautiful thing about kami is its simplicity, which allow the developer full control of the things you mention.
At its core, a kani is two lists. the always messages, and chat history. In between rounds calls you are welcome to modify these lists to control the conversation. For example, I use Kanis to run a multi user multi agent chat/voice conversation. As the conversation moves between different kanis instances and users, I implemented a class that moves parts of chat history between kanis, this way i can maintain context across all the kanis i use. As for extending or managing long conversations, i also purge of the history, however, when i purge i don't just delete, i index purge data so i can be brought back via RAG if the conversation needs it again.