Pocket Flow - LLM Framework in 100 Lines

A 100-line minimalist LLM framework for (Multi-)Agents, Task Decomposition, RAG, etc.

• Install via pip install pocketflow, or just copy the source codes (only 100 lines)

• If the 100 lines feel terse and you’d prefer a friendlier intro, check this out

• 💡 Pro tip!! Build LLM apps with LLMs assistants (ChatGPT, Claude, Cursor.ai, etc.)

  (🫵 Click to expand) Use Claude to build LLM apps

  • Create a project and upload the docs to project knowledge

  • Set project custom instructions. For example:

    1. check "tool.md" and "llm.md" for the required functions.
    2. design the high-level (batch) flow and nodes.
    3. design the shared memory structure: define its fields, data structures, and how they will be updated.
    Think out aloud for above first and ask users if your design makes sense.
    4. Finally, implement. Start with simple, minimalistic codes without, for example, typing.
    

  • Ask it to build LLM apps (Sonnet 3.5 strongly recommended)!

    Help me build a chatbot based on a directory of PDFs.
    

    
  (🫵 Click to expand) Use ChatGPT to build LLM apps

  • Try the GPT assistant. However, it uses older models, which are good for explaining but not that good at coding.

    

  • For stronger coding capabilities, consider sending the docs to more advanced models like O1.

    • Paste the docs link (https://github.com/miniLLMFlow/PocketFlow/tree/main/docs) to Gitingest.

    • Then, paste the generated contents into your O1 prompt, and ask it to build LLM apps.

Documentation: https://minillmflow.github.io/PocketFlow/

Why Pocket Flow?

Pocket Flow is designed to be the framework used by LLMs. In the future, LLM projects will be self-programmed by LLMs themselves: Users specify requirements, and LLMs will design, build, and maintain. Current LLMs are:

1. 👍 Good at Low-level Details: LLMs can handle details like wrappers, tools, and prompts, which don't belong in a framework. Current frameworks are over-engineered, making them hard for humans (and LLMs) to maintain.

2. 👎 Bad at High-level Paradigms: While paradigms like MapReduce, Task Decomposition, and Agents are powerful, LLMs still struggle to design them elegantly. These high-level concepts should be emphasized in frameworks.

The ideal framework for LLMs should (1) strip away low-level implementation details, and (2) keep high-level programming paradigms. Hence, we provide this minimal (100-line) framework that allows LLMs to focus on what matters.

Pocket Flow is also a learning resource, as current frameworks abstract too much away.

How Does it Work?

The 100 lines capture what we see as the core abstraction of most LLM frameworks: a Nested Directed Graph that breaks down tasks into multiple (LLM) steps, with branching and recursion for agent-like decision-making. From there, it’s easy to layer on more complex features.

• To learn more details, please check out documentation: https://minillmflow.github.io/PocketFlow/

• Beginner Tutorial: Text summarization for Paul Graham Essay + QA agent

  • Have questions for this tutorial? Ask LLM assistants through this prompt

• More coming soon ... Let us know you’d love to see!