src/langchain/notebooks/rag_basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generate Answers with Your Documents\n",
    "\n",
    "## Table of Contents\n",
    "- [RAG by Directly Passing Context](#RAG-by-Directly-Passing-Context)\n",
    "- [RAG by Similarity Search](#RAG-by-Similarity-Search)\n",
    "\n",
    "This notebook show an example of Retrieval Augmented Generation that utilizes LangChain to perform question-answering tasks by combining retrieval and generation techniques.\n",
    "\n",
    "We will practice retrieval augmented generation once by passing the context straight to the language model, and once by using FAISS and similarity searching.\n",
    "\n",
    "[FAISS](https://faiss.ai/) is an efficient and high-performance library for vector similarity search. It allows us to find similar things in mountains of data, fast.\n",
    "\n",
    "---\n",
    "\n",
    "## What is Retrieval Augmented Generation?\n",
    "Retrieval Augmented Generation (RAG)'s purpose is to increase the relevance, accuracy and truthfulness of generation. This way we remove the \"data freshness\" problem that LLM's inherently have.\n",
    "\n",
    "It allows you to retrieve relevant documents based on a query and use them as context to generate concise answers to user questions.\n",
    "\n",
    "For more information, see:\n",
    "\n",
    "- [Retrieval Documentation](https://python.langchain.com/docs/modules/data_connection/)\n",
    "\n",
    "- [Advanced Retrieval Types](https://python.langchain.com/docs/modules/data_connection/retrievers/)\n",
    "\n",
    "- [QA with RAG Use Case Documentation](https://python.langchain.com/docs/use_cases/question_answering/)\n",
    "\n",
    "---\n",
    "\n",
    "### Dependencies\n",
    "\n",
    "The Retrieval Augmented Generation module relies on the following dependencies:\n",
    "\n",
    "- `langchain`: The core LangChain library for building the pipeline and running the modules.\n",
    "\n",
    "- `langchain_community`: The LangChain community package that provides additional modules and utilities.\n",
    "\n",
    "- `langchain_core`: The core components and utilities of LangChain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -Uq langchain openai unstructured chardet faiss-gpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Enable LangChain tracing (Optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['LANGSMITH_TRACING_V2'] = \"true\"\n",
    "#os.environ['LANGCHAIN_PROJECT'] = \"unstructuredfileloader\" # Defaults to 'default'\n",
    "os.environ['LANGCHAIN_ENDPOINT'] = \"https://api.smith.langchain.com\"\n",
    "os.environ['LANGCHAIN_API_KEY'] = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RAG by Directly Passing Context\n",
    "For the first RAG example, we will pass the context directly to the language model after splitting it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import chardet\n",
    "\n",
    "from langchain_openai.chat_models import ChatOpenAI\n",
    "from langchain.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_community.document_loaders.unstructured import UnstructuredFileLoader\n",
    "from unstructured.cleaners.core import clean_extra_whitespace\n",
    "\n",
    "# Set API Key\n",
    "os.environ['OPENAI_API_KEY'] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 2: Define the prompt template with the file loader as the context\n",
    "template = \"\"\"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
    "Question: {question} \n",
    "Context: {context} \n",
    "Answer:\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template, context=\"The capital of France is Paris.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_variables=['context', 'question'] messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template=\"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\\nQuestion: {question} \\nContext: {context} \\nAnswer:\\n\"))]\n"
     ]
    }
   ],
   "source": [
    "print(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a LangChain `chain`\n",
    "\n",
    "• A Chain is a sequence of Runnables that are executed in a specific order.\n",
    "\n",
    "• Chains provide a way to string together multiple Runnables to create a workflow or pipeline.\n",
    "\n",
    "• Each Runnable in the Chain takes the output of the previous Runnable as its input.\n",
    "\n",
    "• Chains can be used to build complex applications by combining and orchestrating the execution of multiple Runnables.\n",
    "\n",
    "• They provide a higher-level abstraction for organizing and structuring the flow of data and operations.\n",
    "\n",
    "• Examples of Chains include data processing pipelines, machine learning workflows, and API request/response sequences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 3: Create the LangChain pipeline\n",
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "output_parser = StrOutputParser()\n",
    "\n",
    "chain = (\n",
    "    {\"context\": RunnablePassthrough(), \"question\": RunnablePassthrough()}\n",
    "    | prompt\n",
    "    | llm\n",
    "    | output_parser\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's worth noting that you could easily augment the above code cell to include a `retriever`, but we'll learn more about that later when we test out FAISS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 4: Invoke the LangChain pipeline with a question\n",
    "question = \"What is the capital of France?\"\n",
    "answer = chain.invoke({\"question\": question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The capital of France is Paris.\n"
     ]
    }
   ],
   "source": [
    "print(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explanation\n",
    "In this example, the Retrieval Augmented Generation module is used to answer a user's question about the capital of France. The module retrieves relevant documents based on the query using the `UnstructuredFileLoader` and incorporates them into the prompt template. \n",
    "\n",
    "The LangChain pipeline is then created by chaining together the retriever, prompt, language model (LLM), and output parser components. Finally, the pipeline is invoked with the user's question, and the generated answer is printed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Customization\n",
    "\n",
    "The Retrieval Augmented Generation module can be customized to fit your specific needs. Here are some areas to consider:\n",
    "\n",
    "• Modify the prompt template to structure the prompt according to your requirements. You can include placeholders for the question, retrieved context, or any other information you want to provide to the language model.\n",
    "\n",
    "• Use different language models by specifying the desired model name when creating the `ChatOpenAI` instance. You can explore different models provided by OpenAI, open-source models on the HuggingFace Hub, or use your own fine-tuned models.\n",
    "\n",
    "• Customize the output parser to parse the generated answer in a format that suits your application's needs.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RAG by Similarity Search\n",
    "Now let's move on to the second example, using FAISS and similarity search. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Documents\n",
    "\n",
    "We'll first need some new context to test out. Let's try using the `WebBaseLoader`, a flexible loader for online content. It works by loading HTML pages using urllib and parsing them with BeautifulSoup.\n",
    "\n",
    "See the [API Documentation](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.web_base.WebBaseLoader.html?highlight=webbaseloader#langchain_community.document_loaders.web_base.WebBaseLoader) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://python.langchain.com/docs/expression_language/\")\n",
    "\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "len(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split documents\n",
    "\n",
    "Previously, when working to pass context directly to the LLM, we didn't split our context into chunks. However, because we're going to perform similarity searching using a FAISS index, we need to split our documents into chunks.\n",
    "\n",
    "That way when we enter a query, only the most relevant information from the index will be passed in as context.\n",
    "\n",
    "### Split Data into Chunks with `RecursiveCharacterTextSplitter`\n",
    "\n",
    "To make effective use of our loaded documents (files) we need to split them into manageable chunks.\n",
    "\n",
    "Generally speaking, smaller chunks warrant more accurate results, but may take longer to process.\n",
    "\n",
    "### Go Deeper\n",
    "\n",
    "#### Accuracy with Smaller Chunks\n",
    "* **Increased Focus**: Smaller chunks of text or queries allow the system to focus on a more specific set of information. This specificity can lead to more accurate and relevant results because the system is not overwhelmed by too much or too broad information.\n",
    "* **Contextual Relevance**: With a narrower focus, the likelihood of retrieving information that is contextually relevant to more specific queries, enhancing the accuracy of the response.\n",
    "\n",
    "#### Processing Time\n",
    "* **Multiple Queries**: Smaller chunks might require multiple queries to cover a topic comprehensively. Each query involves a separate retrieval process, which cumulatively can take more time.\n",
    "* **Trade-off Between Depth and Breadth**: While smaller queries allow for a depth in a specific area, they might necessitate multiple rounds of retrieval to get a broad understanding, thus increasing overall processing time.\n",
    "\n",
    "#### System Limitations and Efficiency:\n",
    "* **Computational Load**: Smaller chunks means more frequent calls to the retrieval system. Depending on the efficiency of the system, this can either slow down the process due to computational load or, if the system is highly efficient, might not significantly impact the processing time.\n",
    "\n",
    "The following cell demonstrates how to split the loaded data into chunks using the Langchain library. We'll instantiate a variable, `text_splitter`, with the `RecursiveCharacterTextSplitter` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=0)\n",
    "documents = text_splitter.split_documents(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "len(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index Documents\n",
    "\n",
    "Next we'll index the documents using FAISS. The following cell, you can practice saving and loading your FAISS index locally, but it's not a necessary step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAIEmbeddings\n",
    "from langchain_community.vectorstores import FAISS\n",
    "\n",
    "embeddings = OpenAIEmbeddings()\n",
    "# Initialize FAISS as our vector database and embed our documents inside\n",
    "vector = FAISS.from_documents(documents, embeddings)\n",
    "# Now our vector database is ready to be queried"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the index to disk\n",
    "vector.save_local(\"test_index\")\n",
    "\n",
    "# Load the index from disk\n",
    "load_local_vector = FAISS.load_local(\"test_index\", embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying Documents\n",
    "\n",
    "### Similarity Searching for relevant context from a swath of data\n",
    "\n",
    "We'll be performing two different types of searching against our index, first by using a basic, and subsequently a more advanced approach.\n",
    "\n",
    "## ***Basic Retrieval***:\n",
    "\n",
    "Before we do retrieval, let's set up a prompt template we can easily pass data through when the user asks a question. We'll do this with `ChatPromptTemplate` and then pass it into `create_stuff_documents_chain`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "# Create a prompt template\n",
    "prompt = ChatPromptTemplate.from_template(\"\"\"Answer the following question based only on the provided context:\n",
    "\n",
    "<context>\n",
    "{context}\n",
    "</context>\n",
    "\n",
    "Question: {input}\"\"\")\n",
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\")\n",
    "\n",
    "document_chain = create_stuff_documents_chain(llm, prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import create_retrieval_chain\n",
    "\n",
    "# Use our Vector Database as our `Retriever`\n",
    "retriever = vector.as_retriever()\n",
    "\n",
    "# Create our basic retrieval chain\n",
    "retrieval_chain = create_retrieval_chain(retriever, document_chain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Invoke the basic chain\n",
    "response = retrieval_chain.invoke(\n",
    "    {\"input\": \"how can langsmith help with testing?\"}\n",
    "    )\n",
    "print(response[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ***Advanced Retrieval***:\n",
    "\n",
    "We'll use the `MultiQueryRetriever` class to query our index using multiple queries.\n",
    "This specific retriever works by being given a query, and using an LLM to write a set of queries for more specific results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.retrievers import MultiQueryRetriever\n",
    "\n",
    "# Pass in `retriever` directly, NOT wrappers `retrieval_chain` or `response`\n",
    "advanced_retriever = MultiQueryRetriever.from_llm(retriever=retriever, llm=llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create our advanced retrieval chain\n",
    "retrieval_chain = create_retrieval_chain(advanced_retriever, document_chain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LangSmith can help with testing by allowing users to quickly edit examples and add them to datasets to expand the evaluation sets, fine-tune a model for improved quality or reduced costs, monitor application performance, log all traces, visualize latency and token usage statistics, troubleshoot specific issues, and extract insights from logged runs. Additionally, LangSmith simplifies the approach to constructing datasets and provides examples for integrating with third parties for testing purposes.\n"
     ]
    }
   ],
   "source": [
    "# Invoke the advanced chain\n",
    "response = retrieval_chain.invoke(\n",
    "    {\"input\": \"how can langsmith help with testing?\"}\n",
    "    )\n",
    "print(response[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "The Retrieval Augmented Generation module provides a convenient way to perform question-answering tasks by combining retrieval and generation techniques. It allows you to retrieve relevant documents based on a query and generate concise answers using LangChain. With its customization options, you can tailor the module to suit your specific needs and integrate it into your applications for enhanced question-answering capabilities."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}