Skip to content

Added pebblo safe rag demo using google drive , qdrant, hugging face and groq api #599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 6, 2025

Conversation

gr8nishan
Copy link
Collaborator

@gr8nishan gr8nishan commented Jun 6, 2025

Added Pebblo SafeRAG Demo Implementation

Overview

Implemented a secure Retrieval-Augmented Generation (RAG) demo that combines Pebblo's security features with modern open source tools like HuggingFace, Llama model using Groq, Qdrant, and Langchain

Key Components Added

1. Document Ingestion & Security

  • Integrated Google Drive as document source
  • Implemented PebbloSafeLoader for semantic filtering and access control
  • Added identity-based access control for document retrieval

2. Vector Store & Embeddings

  • Set up Qdrant vector database for efficient document storage and retrieval
  • Integrated HuggingFace local embeddings for semantic search
  • Implemented document chunking and vectorization pipeline

3. LLM Integration

  • Added Groq API integration with Llama 3.3 model
  • Implemented secure query processing
  • Added semantic filtering for sensitive content

Technical Implementation

  • Used LangChain for RAG pipeline orchestration
  • Implemented local embedding generation for privacy
  • Added interactive query interface
  • Set up environment-based configuration

@gr8nishan gr8nishan changed the title Added demo with all open source components Added demo for safe rag using google drive , qdrant, hugging face and froq api Jun 6, 2025
@gr8nishan gr8nishan changed the title Added demo for safe rag using google drive , qdrant, hugging face and froq api Added pebblo safe rag demo using google drive , qdrant, hugging face and froq api Jun 6, 2025
@gr8nishan gr8nishan changed the title Added pebblo safe rag demo using google drive , qdrant, hugging face and froq api Added pebblo safe rag demo using google drive , qdrant, hugging face and groq api Jun 6, 2025
Copy link
Collaborator

@srics srics left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gr8nishan gr8nishan merged commit dd92db4 into daxa-ai:main Jun 6, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants