Chunking .py .pdf .txt files from Working directory for reading and prompting.

[emskiemre]

For now Auto-Gpt seems to summerize and chunk webcontent or web pages before sending a request to OpenAI as we have a token limit. But it does not chunk large .txt or .py files when it tries to read them using read_file command making the used token size very large before sending a request to OpenAI.

Can this be fixed ?

[ChessScholar]

Nice timing, I just made a post about that with a summarizing technique. It doesn't help with the sending of tokens, but it sure helps with the receiving of them.
1061

[emskiemre]

Nice post !

Is there a way to achieve this ?
So it can only use the GPt-3.5 model like for evaluate_code by summerizing or chunking a large .py file ?