[Bug] Forking Breaks If Over 150k Tokens

[ColbySerpa]

You can reproduce this bug easily with Claude if your original chat is over 150k tokens and then you fork it. In the fork, as you approach 170k and 180k tokens, every message you send will fail because it says that the 200k token limit has been reached. For some reason, the token counting is inaccurate in the fork.

To get around it, I have to delete the fail message and press retry — then it will work. Every single time I have to send a message in the fork I have to delete the error reply message and then retry which is incredibly annoying and slows me down. It would be great if the forking feature was fixed.

If you have any questions please let me know.

[lobehubbot]

👀 @ColbySerpa

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot]

Hey @ColbySerpa! I'm here to help you with any bugs, questions, or contributions. Let's tackle this issue together!

The issue you're experiencing with token counting in forked chats might be related to how tokens are calculated and managed in the framework. The token counting logic is implemented in the src/features/ChatInput/ActionBar/Token/TokenTag.tsx file, where various components like input, messageString, systemRole, historySummary, and toolsString are used to compute the total token count ^[1].

Additionally, the getMessagesTokenCount function in src/store/chat/helpers.ts calculates the token count for a list of messages by encoding their content ^[2]. There are tests for this function to ensure it returns the correct token count, but there is no specific mention of tests related to token management in forked chats ^[3].

The error you're encountering might be due to a discrepancy in how tokens are counted in forked chats compared to the original chat. It could be beneficial to review the token counting logic in these files to identify any potential issues or inaccuracies specific to forked chats.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[arvinxx]

it seems a bug, I will figure it out

[ColbySerpa]

it seems a bug, I will figure it out

@arvinxx Thank you so much! It will make my life 1000x easier. 🙏