A modular multi-agent architecture for evaluating hallucination reduction strategies in large language models (LLMs) using structured planning taxonomies, agent-level reasoning, and semantic similarity metrics.
This system combines agent generation, response storage, planning categorization, and vector-based evaluation using FAISS, PCA, and KMeans clustering — implemented and tested on a local TinyLLaMA server.
📘 For a detailed visual walkthrough, check out the full Canva presentation here:
This project explores hallucination mitigation in LLMs by:
- Building a planning-aware agent architecture Future work - profiling, memory, tools, and action components
- Storing and indexing responses using semantic vector search (FAISS)
- Classifying agent strategies using a planning taxonomy
- Measuring response stability and alignment via embedding-based pairwise analysis
- Clustering performance using PCA + KMeans
- Evaluating complexity vs. performance tradeoffs for each agent
| Block | Description |
|---|---|
| A | LLM setup, calling multiple planning agent, storing agent responses in a vector Database (FAISS), setting up structred database and semantic search system for agent evaluation |
| B | Multi-agent response evaluation using cosine & euclidean distances, response length, completion time as features/metrics |
| C | PCA-reduced Vector database and pairwise score analysis - interpretable evaluation though principles of triangulation |
| D | Heatmaps of pairwise agent similarity |
| E | Per-agent aggregated metrics across tasks |
| F | KMeans clustering of agent pair performance |
| G | Planning taxonomy–aware clustering |
| H | Complexity-based agent ranking |
Each block is modular and reproducible.
Information Model Overview – The Information Model: Agent selection and weighting framework across task, category, and library matrices. Supports feedback-based learning and triangulation-based reasoning. Click here to know more
git clone https://github.com/GauraangMalikk/Information-Model-an-LLM-Agent-Architecture.git
cd Information-Model-an-LLM-Agent-Architecture