Skip to content

kaizengrowth/CityCamp_AI

Repository files navigation

πŸ›οΈ CivicSpark AI - Tulsa Civic Engagement Platform

A comprehensive CivicTech platform connecting Tulsa residents with city government through AI-powered tools, automated notifications, and intelligent meeting analytics.

πŸ“Š Project Status & Info

Deployment Status Security License

GitHub last commit GitHub issues GitHub stars

Tech Stack Code Size Dependencies Contributions Welcome

🌐 Deployed Demo: https://d1s9nkkr0t3pmn.cloudfront.net

CivicSpark AI Homepage

🌟 Features

πŸ€– AI-Powered Civic Assistant

  • Interactive chatbot with real-time city council knowledge
  • RAG-Enhanced Responses: Document-based answers using city budgets, legislation, and policies
  • Natural language queries about Tulsa government with contextual document search
  • Meeting summary generation and analysis

πŸ“… Smart Meeting Notifications

  • Automated alerts for city council meetings
  • Topic-based subscriptions (housing, transportation, etc.)
  • SMS and email delivery with AI-categorized content

πŸ“Š Intelligent Meeting Analytics

  • AI categorization of 42+ civic topics
  • Automated agenda extraction and impact assessment
  • Searchable meeting minutes with keyword analysis

πŸ’¬ Representative Communication

  • AI-powered email generation to contact officials
  • District-based representative lookup
  • Pre-built templates for common civic issues

πŸ—³οΈ Community Engagement

  • Campaign tracking and petition management
  • Neighborhood-based organizing tools
  • User preference and notification management

πŸ“„ RAG System (Retrieval-Augmented Generation)

🎯 What is RAG?

The RAG system enables the chatbot to search and reference actual city documents, budgets, legislation, and policies to provide accurate, source-backed answers instead of generic responses.

πŸ”§ Key Components

  • Vector Database: ChromaDB (dev) / FAISS (production) for semantic search
  • Document Processing: AI-powered text extraction, chunking, and summarization
  • Embedding Generation: OpenAI text-embedding-3-small for vector representations
  • Smart Search: Contextual document retrieval with relevance scoring

πŸ“ Supported Document Types

  • πŸ’° Budgets: City financial documents and allocations
  • πŸ“œ Legislation: Ordinances, resolutions, and policies
  • πŸ“‹ Meeting Minutes: City council and committee records
  • πŸ“Š Reports: Studies, analyses, and official reports
  • πŸ“ Policies: Administrative procedures and guidelines

πŸš€ Quick RAG Setup

# Install RAG dependencies
pip install -r backend/requirements.txt

# Run database migration
cd backend && python -m alembic upgrade head

# Test the system
python scripts/test_rag_system.py

πŸ“– Full Documentation: See docs/RAG_SYSTEM_README.md

πŸ§ͺ LLM-as-Judge Evaluation System

🎯 Intelligent Chatbot Testing

Advanced evaluation framework using GPT-4 to assess chatbot response quality beyond traditional keyword matching.

πŸ”§ Key Features

  • Smart Quality Assessment: GPT-4 evaluates helpfulness, accuracy, and civic appropriateness
  • Semantic Understanding: Goes beyond keywords to understand context and meaning
  • Comparative Analysis: Traditional metrics vs. LLM evaluation with combined scoring
  • Actionable Feedback: Specific improvement suggestions and response strengths

πŸ“Š Evaluation Metrics

  • Accuracy: Factual correctness and Tulsa civic relevance
  • Helpfulness: How well responses assist users with civic questions
  • Completeness: Information sufficiency without verbosity
  • Civic Appropriateness: Tone and context suitability for government use

πŸš€ Quick Evaluation

# LLM-as-Judge evaluation
python scripts/run_eval.py --mode llm-judge

# Compare traditional vs LLM metrics
python scripts/llm_judge_evaluator.py

# Performance benchmarking
python scripts/run_eval.py --mode performance

πŸ“ˆ Sample Results

LLM-AS-JUDGE EVALUATION SUMMARY
Combined Score: 0.847/1.0 (Grade: B)
Traditional Score: 0.789/1.0
LLM Judge Score: 0.873/1.0
Score Improvement: +0.084

Grade Distribution: A: 2, B: 6, C: 2, D: 0, F: 0

πŸ“– Full Documentation: See docs/CHATBOT_EVALUATION_README.md

πŸš€ Quick Start

Local Development

# Clone repository
git clone https://github.com/kaizengrowth/CityCamp_AI.git
cd CityCamp_AI

# Option 1: Automated setup (recommended)
./scripts/start-dev.sh

# Option 2: Manual setup
# Backend
cd backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp env.example .env
python -m app.main

# Frontend (new terminal)
cd frontend
npm install
npm run dev

Access Points:

Production Access

πŸ—οΈ Architecture

Technology Stack

  • Frontend: React 18 + TypeScript + Vite + Tailwind CSS
  • Backend: FastAPI + Python 3.11 + PostgreSQL + Redis
  • AI/ML: OpenAI GPT-4 + Custom categorization models + RAG System (ChromaDB/FAISS)
  • Document Processing: Multi-format support (PDF, DOCX, TXT) with vector embeddings
  • Evaluation: LLM-as-Judge system with GPT-4 intelligent response assessment
  • Infrastructure: AWS (ECS Fargate, RDS, ElastiCache, S3, CloudFront)
  • CI/CD: GitHub Actions with automated testing and deployment

Comprehensive System Architecture

The application follows a layered architecture with clear separation of concerns, dependency injection, and standardized patterns across all components.

πŸ“Š System Architecture Overview

graph TB
    subgraph "🌐 Frontend"
        FE[React + TypeScript SPA]
    end

    subgraph "πŸš€ API Layer"
        API[FastAPI Backend]
        ENDPOINTS[REST Endpoints]
        API --> ENDPOINTS
    end

    subgraph "βš™οΈ Services"
        CHAT[AI Chatbot Service]
        RAG[RAG Document Search]
        NOTIF[Notification Service]
        SCRAPER[Meeting Scrapers]
    end

    subgraph "πŸ’Ύ Data Layer"
        DB[(PostgreSQL)]
        VECTOR[(Vector Store)]
        CACHE[(Redis Cache)]
        S3[(File Storage)]
    end

    subgraph "πŸ€– External APIs"
        OPENAI[OpenAI GPT-4]
        TWILIO[Twilio SMS]
    end

    %% Main connections
    FE --> API
    API --> CHAT
    API --> RAG
    API --> NOTIF
    API --> SCRAPER

    CHAT --> OPENAI
    RAG --> VECTOR
    NOTIF --> TWILIO
    SCRAPER --> DB

    API --> DB
    API --> CACHE
    API --> S3
Loading

🎯 Key Architecture Components:

  • Frontend: React 18 SPA with TypeScript for type safety
  • API Layer: FastAPI with standardized responses and error handling
  • AI Services: GPT-4 chatbot with RAG document search capabilities
  • Data Storage: PostgreSQL + Vector store + Redis caching + S3 files
  • External Integrations: OpenAI for AI, Twilio for SMS notifications

πŸ”§ Key Architectural Patterns

  • Dependency Injection: Services use constructor injection for better testability
  • Base Service Pattern: All services inherit from BaseService for consistency
  • Standardized Responses: Unified API response formats with proper error handling
  • Unified Data Models: Consolidated notification preferences and user management
  • RAG Architecture: Document processing with vector embeddings for semantic search
  • Event-Driven Notifications: Async notification processing with queue management

πŸ“Š Detailed Service Dependencies

graph TB
    subgraph "🌐 Frontend Layer"
        FE[React 18 + TypeScript SPA]
        FE_COMP[Components Layer]
        FE_PAGES[Pages Layer]
        FE_CTX[Contexts Layer]
        FE_API[API Config]

        FE --> FE_COMP
        FE --> FE_PAGES
        FE --> FE_CTX
        FE --> FE_API
    end

    subgraph "πŸ”Œ API Gateway Layer"
        CF[CloudFront CDN]
        ALB[Application Load Balancer]
        CORS[CORS Middleware]

        CF --> ALB
        ALB --> CORS
    end

    subgraph "πŸš€ FastAPI Application"
        MAIN[main.py - App Entry]
        ROUTER[API Router v1]
        MIDDLEWARE[Middleware Stack]
        EXCEPTION[Exception Handlers]

        MAIN --> ROUTER
        MAIN --> MIDDLEWARE
        MAIN --> EXCEPTION
    end

    subgraph "πŸ“‘ API Endpoints"
        EP_AUTH[Auth Endpoints]
        EP_MEET[Meetings Endpoints]
        EP_ORG[Organizations Endpoints]
        EP_CAMP[Campaigns Endpoints]
        EP_CHAT[Chatbot Endpoints]
        EP_DOC[Documents Endpoints]
        EP_SUB[Subscriptions Endpoints]
        EP_REP[Representatives Endpoints]

        ROUTER --> EP_AUTH
        ROUTER --> EP_MEET
        ROUTER --> EP_ORG
        ROUTER --> EP_CAMP
        ROUTER --> EP_CHAT
        ROUTER --> EP_DOC
        ROUTER --> EP_SUB
        ROUTER --> EP_REP
    end

    subgraph "βš™οΈ Service Layer"
        BASE_SVC[BaseService Abstract Class]
        CHAT_SVC[ChatbotService]
        DOC_SVC[DocumentProcessingService]
        VEC_SVC[VectorService]
        NOTIF_SVC[NotificationService]
        TWILIO_SVC[TwilioService]
        S3_SVC[S3Service]
        GEO_SVC[GeocodingService]
        AUTH_SVC[AuthService]
        RESEARCH_SVC[ResearchService]

        BASE_SVC -.-> CHAT_SVC
        BASE_SVC -.-> DOC_SVC
        BASE_SVC -.-> VEC_SVC
        BASE_SVC -.-> NOTIF_SVC
        BASE_SVC -.-> TWILIO_SVC
        BASE_SVC -.-> S3_SVC
        BASE_SVC -.-> GEO_SVC
        BASE_SVC -.-> RESEARCH_SVC

        CHAT_SVC --> VEC_SVC
        CHAT_SVC --> RESEARCH_SVC
        DOC_SVC --> VEC_SVC
        NOTIF_SVC --> TWILIO_SVC
    end

    subgraph "πŸ—„οΈ Data Models Layer"
        USER_MODEL[User Model]
        MEET_MODEL[Meeting Model]
        ORG_MODEL[Organization Model]
        CAMP_MODEL[Campaign Model]
        DOC_MODEL[Document Model]
        NOTIF_PREF[NotificationPreferences Model]
        SUB_MODEL[Subscription Models]

        NOTIF_PREF --> USER_MODEL
    end

    subgraph "πŸ’Ύ Data Storage Layer"
        POSTGRES[(PostgreSQL Database)]
        REDIS[(Redis Cache)]
        CHROMA[(ChromaDB Vector Store)]
        FAISS[(FAISS Vector Index)]
        S3[(AWS S3 Storage)]

        VEC_SVC --> CHROMA
        VEC_SVC --> FAISS
        S3_SVC --> S3
    end

    subgraph "πŸ€– AI & External Services"
        OPENAI[OpenAI GPT-4 API]
        EMBEDDING[OpenAI Embeddings]
        TWILIO_API[Twilio SMS API]
        GOOGLE_API[Google Search API]
        GEOCODING[Geocoding APIs]

        CHAT_SVC --> OPENAI
        VEC_SVC --> EMBEDDING
        TWILIO_SVC --> TWILIO_API
        RESEARCH_SVC --> GOOGLE_API
        GEO_SVC --> GEOCODING
    end

    subgraph "πŸ” Data Collection"
        SCRAPERS[Meeting Scrapers]
        TGOV_SCRAPER[TGOV Scraper]
        ARCHIVE_SCRAPER[Archive Scraper]

        SCRAPERS --> TGOV_SCRAPER
        SCRAPERS --> ARCHIVE_SCRAPER
    end

    subgraph "☁️ AWS Infrastructure"
        ECS[ECS Fargate]
        RDS[RDS PostgreSQL]
        ELASTICACHE[ElastiCache Redis]
        CLOUDFRONT[CloudFront CDN]

        ECS --> RDS
        ECS --> ELASTICACHE
        CLOUDFRONT --> ECS
    end

    subgraph "πŸ“Š Response Standardization"
        STD_LIST[StandardListResponse]
        STD_RESP[StandardResponse]
        ERR_RESP[ErrorResponse]
        PAGINATION[PaginationParams]

        EP_MEET --> STD_LIST
        EP_ORG --> STD_LIST
        EP_CAMP --> STD_LIST
        EP_DOC --> STD_LIST
    end

    %% Connections
    FE_API --> CF
    CF --> MAIN

    EP_AUTH --> AUTH_SVC
    EP_MEET --> CHAT_SVC
    EP_ORG --> BASE_SVC
    EP_CAMP --> BASE_SVC
    EP_CHAT --> CHAT_SVC
    EP_DOC --> DOC_SVC
    EP_SUB --> NOTIF_SVC

    USER_MODEL --> POSTGRES
    MEET_MODEL --> POSTGRES
    ORG_MODEL --> POSTGRES
    CAMP_MODEL --> POSTGRES
    DOC_MODEL --> POSTGRES
    NOTIF_PREF --> POSTGRES
    SUB_MODEL --> POSTGRES

    SCRAPERS --> MEET_MODEL

    POSTGRES --> RDS
    REDIS --> ELASTICACHE
    S3 --> S3
    CF --> CLOUDFRONT
Loading

🎯 Architectural Principles

  • Single Responsibility: Each service has a focused, well-defined purpose
  • Dependency Inversion: High-level modules don't depend on low-level modules
  • Interface Segregation: Clients depend only on interfaces they use
  • Open/Closed: Open for extension, closed for modification
  • DRY (Don't Repeat Yourself): Shared functionality in base classes
  • Consistent Error Handling: Standardized exception handling across all layers
  • Type Safety: Full TypeScript and Python type coverage

πŸ”„ Recent Architectural Improvements

The codebase has been significantly refactored to improve maintainability and consistency:

βœ… Standardization Improvements:

  • Unified Response Format: All API endpoints now use StandardListResponse and PaginationParams
  • Base Service Pattern: All services inherit from BaseService with dependency injection
  • Centralized Error Handling: Custom exception classes with structured error responses
  • Unified Notification System: Consolidated user and subscription notification preferences

βœ… Code Quality Improvements:

  • Reduced Duplication: Eliminated ~200 lines of redundant code
  • Better Type Safety: Full type coverage with generic response types
  • Improved Testability: Dependency injection enables easier mocking
  • Consistent Logging: Standardized logging patterns across all services

βœ… Database Improvements:

  • Unified Models: NotificationPreferences model consolidates user notification settings
  • Proper Migrations: Alembic migrations preserve existing data during schema changes
  • Better Relationships: Cleaner model relationships with proper foreign keys

See docs/ARCHITECTURE_IMPROVEMENTS.md for detailed implementation notes.

πŸ“ Project Structure

CityCamp_AI/
β”œβ”€β”€ 🎨 frontend/              # React TypeScript application
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ components/       # React components
β”‚   β”‚   β”œβ”€β”€ pages/           # Route components
β”‚   β”‚   β”œβ”€β”€ contexts/        # React contexts
β”‚   β”‚   └── config/          # API configuration
β”‚   β”œβ”€β”€ package.json         # Dependencies
β”‚   └── vite.config.ts       # Vite configuration
β”‚
β”œβ”€β”€ βš™οΈ backend/               # FastAPI Python backend
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ api/v1/          # API endpoints with standardized responses
β”‚   β”‚   β”‚   └── endpoints/   # Individual endpoint modules
β”‚   β”‚   β”œβ”€β”€ core/            # Core application components
β”‚   β”‚   β”‚   β”œβ”€β”€ config.py    # Application configuration
β”‚   β”‚   β”‚   β”œβ”€β”€ database.py  # Database connection and setup
β”‚   β”‚   β”‚   └── exceptions.py # Custom exception handling
β”‚   β”‚   β”œβ”€β”€ models/          # SQLAlchemy database models
β”‚   β”‚   β”‚   β”œβ”€β”€ user.py      # User and authentication models
β”‚   β”‚   β”‚   β”œβ”€β”€ meeting.py   # Meeting and agenda models
β”‚   β”‚   β”‚   β”œβ”€β”€ organization.py # Community organization models
β”‚   β”‚   β”‚   β”œβ”€β”€ campaign.py  # Civic campaign models
β”‚   β”‚   β”‚   β”œβ”€β”€ document.py  # RAG document models
β”‚   β”‚   β”‚   └── notification_preferences.py # Unified notification system
β”‚   β”‚   β”œβ”€β”€ schemas/         # Pydantic schemas for API
β”‚   β”‚   β”‚   β”œβ”€β”€ base.py      # Standardized response schemas
β”‚   β”‚   β”‚   └── *.py         # Individual endpoint schemas
β”‚   β”‚   β”œβ”€β”€ services/        # Business logic layer
β”‚   β”‚   β”‚   β”œβ”€β”€ base.py      # BaseService abstract class
β”‚   β”‚   β”‚   β”œβ”€β”€ chatbot_service.py # AI chatbot with RAG
β”‚   β”‚   β”‚   β”œβ”€β”€ vector_service.py  # Vector embeddings and search
β”‚   β”‚   β”‚   β”œβ”€β”€ document_processing_service.py # Document processing
β”‚   β”‚   β”‚   β”œβ”€β”€ notification_service.py # Meeting notifications
β”‚   β”‚   β”‚   └── *.py         # Other specialized services
β”‚   β”‚   └── scrapers/        # Data collection modules
β”‚   β”œβ”€β”€ alembic/            # Database migrations
β”‚   β”œβ”€β”€ requirements.txt     # Python dependencies
β”‚   └── main.py             # Application entry point with exception handlers
β”‚
β”œβ”€β”€ ☁️ aws/                   # Infrastructure as Code
β”‚   β”œβ”€β”€ terraform/           # Terraform configurations
β”‚   β”œβ”€β”€ scripts/             # Deployment scripts
β”‚   └── README.md           # AWS deployment guide
β”‚
β”œβ”€β”€ πŸ“š docs/                  # Comprehensive documentation
β”‚   β”œβ”€β”€ QUICKSTART.md        # 5-minute setup guide
β”‚   β”œβ”€β”€ RAG_SYSTEM_README.md # RAG architecture & usage guide
β”‚   β”œβ”€β”€ CHATBOT_EVALUATION_README.md # LLM-as-Judge evaluation guide
β”‚   β”œβ”€β”€ ARCHITECTURE_IMPROVEMENTS.md # Recent refactoring details
β”‚   β”œβ”€β”€ aws-deployment-guide.md
β”‚   β”œβ”€β”€ TROUBLESHOOTING.md   # Issue resolution
β”‚   └── API_DOCUMENTATION.md # API reference
β”‚
β”œβ”€β”€ πŸ§ͺ tests/                 # Centralized testing
β”‚   β”œβ”€β”€ backend/             # Python API tests
β”‚   β”œβ”€β”€ frontend/            # React component tests
β”‚   └── README.md           # Testing instructions
β”‚
└── πŸ”§ scripts/               # Development & deployment
    β”œβ”€β”€ start-dev.sh         # Local development setup
    β”œβ”€β”€ eval_chatbot_system.py # Traditional evaluation system
    β”œβ”€β”€ llm_judge_evaluator.py # LLM-as-Judge evaluation
    β”œβ”€β”€ run_eval.py          # Evaluation runner with multiple modes
    β”œβ”€β”€ test_rag_system.py   # RAG system testing
    β”œβ”€β”€ test_production_api.sh # Production diagnostics
    └── fix_production_api.sh  # Production issue fixes

πŸ”§ Development

Environment Setup

# Check requirements
node --version  # >= 18.0.0
python --version # >= 3.11.0
docker --version # Latest

# Setup with automation
./scripts/start-dev.sh

# Manual database setup (if needed)
./scripts/setup_database.py

Key Development Commands

# Frontend
cd frontend
npm run dev          # Start development server
npm run build        # Build for production
npm run test         # Run component tests
npm run lint         # Code linting

# Backend
cd backend
source venv/bin/activate
python -m app.main   # Start development server
python -m pytest ../tests/backend/ -v  # Run API tests
python -m app.services.meeting_scraper  # Live meeting discovery

# Full system
./scripts/start-dev.sh     # Start all services
./scripts/test-all.sh      # Run all tests

πŸš€ Production Deployment

Automated Deployment

# Deploy to AWS (requires AWS CLI configured)
./aws/scripts/deploy.sh

# Check production health
./scripts/test_production_api.sh

# Fix production issues
./scripts/fix_production_api.sh

Manual Deployment Steps

  1. Infrastructure: Deploy with Terraform (aws/terraform/)
  2. Backend: ECS Fargate service with auto-scaling
  3. Frontend: S3 + CloudFront distribution
  4. Database: RDS PostgreSQL with automated backups
  5. Monitoring: CloudWatch logs and alerts

πŸ“Š System Status & Health

Performance Metrics

  • πŸ“ˆ API Response Time: < 500ms average
  • πŸ“Š Database: 40+ meetings with full AI categorization
  • πŸ”„ Uptime: 99%+ availability
  • πŸ’Ύ Data: AI-processed meeting minutes and agenda items

πŸ§ͺ Testing

Run All Tests

# Automated testing
npm run test           # Frontend unit tests
python -m pytest tests/backend/ -v  # Backend API tests

# Manual testing
./scripts/test_production_api.sh  # Production diagnostics
curl http://localhost:8000/health # Health checks

Test Coverage

  • βœ… API Endpoints: All CRUD operations
  • βœ… Database Models: Meeting, User, Notification schemas
  • βœ… AI Services: Text extraction and categorization
  • βœ… Frontend Components: Key UI components
  • βœ… Production Health: End-to-end API testing

πŸ› οΈ Troubleshooting

Common Issues & Quick Fixes

Meeting Details Not Loading:

./scripts/fix_production_api.sh  # Automated fix

Backend Won't Start:

lsof -ti:8000 | xargs kill -9  # Kill conflicting processes
cd backend && source venv/bin/activate && python -m app.main

Frontend Dependencies Issues:

cd frontend && rm -rf node_modules package-lock.json && npm install

Production API Issues:

./scripts/test_production_api.sh  # Diagnose issues
./scripts/fix_production_api.sh   # Apply fixes

More solutions: See docs/TROUBLESHOOTING.md

πŸ“š Documentation

Getting Started

Deployment & Operations

API & Development

🀝 Contributing

Development Workflow

  1. Fork the repository
  2. Clone your fork: git clone https://github.com/YOUR_USERNAME/CityCamp_AI.git
  3. Setup: Run ./scripts/start-dev.sh
  4. Develop: Make changes and test locally
  5. Test: Run npm test and python -m pytest tests/backend/ -v
  6. Submit: Create pull request

Code Standards

  • βœ… TypeScript: Strict mode enabled
  • βœ… Python: Type hints and docstrings
  • βœ… Testing: Unit tests for new features
  • βœ… Linting: Pre-commit hooks configured
  • βœ… Documentation: Update README for new features

πŸ“Š Monitoring & Analytics

Production Monitoring

  • πŸ“ˆ CloudWatch: Application and infrastructure metrics
  • πŸ” Logs: Centralized logging with search capabilities
  • 🚨 Alerts: Automated notifications for issues
  • πŸ“Š Dashboard: Real-time system health metrics

Key Metrics

  • 🌐 Web Traffic: CloudFront analytics
  • πŸ“± API Usage: Request volume and response times
  • πŸ’Ύ Database: Query performance and connection health
  • πŸ€– AI Services: API usage and categorization accuracy

πŸ” Security

Security Features

  • πŸ›‘οΈ HTTPS: All communication encrypted
  • πŸ” Environment Variables: Secrets management
  • πŸ—οΈ VPC: Network isolation in AWS
  • πŸ‘€ IAM: Minimal permission roles
  • πŸ” Security Scanning: Automated vulnerability checks

Data Protection

  • πŸ“Š Database Encryption: At rest and in transit
  • πŸ”’ API Authentication: JWT tokens
  • πŸ›‘οΈ Input Validation: XSS and injection prevention
  • πŸ“‹ Audit Logs: Comprehensive activity tracking

πŸ“„ License

MIT License - see LICENSE file for details.

πŸ†˜ Support

Get Help

  • πŸ“š Documentation: Check docs/ directory
  • πŸ› Issues: Create GitHub issue with details
  • πŸ’¬ Discussions: Use GitHub Discussions for questions
  • πŸ› οΈ Emergency: Run ./scripts/fix_production_api.sh for production issues

Contact


πŸ“ˆ Recent Updates

RAG System & Evaluation Architecture Changes (August 5, 2025):

  • βœ… Implemented LLM-as-Judge evaluation system using GPT-4 for intelligent response assessment
  • βœ… Comprehensive RAG system for document-based chatbot responses with vector search
  • βœ… Added advanced evaluation framework with traditional + LLM metrics comparison
  • βœ… Created document processing pipeline with multi-format support (PDF, DOCX, TXT)
  • βœ… Integrated RAG into existing chatbot with OpenAI function calling
  • βœ… Added complete document management API with upload/search endpoints
  • βœ… Built evaluation suite with multiple modes (quick, full, RAG, performance, LLM-judge)
  • βœ… Fixed production meeting details loading issues
  • βœ… Added comprehensive production troubleshooting scripts
  • βœ… Resolved security vulnerabilities (esbuild, dependencies)
  • βœ… Enhanced CI/CD pipeline with automated testing
  • βœ… Updated documentation and README files

Next Planned Features:

  • πŸ“„ RAG System Expansion: Upload city budgets, ordinances, and legislation
  • πŸ” Advanced Document Search: Semantic search across all city documents
  • πŸ§ͺ Evaluation System Enhancement: A/B testing, user satisfaction metrics, continuous monitoring
  • πŸ€– Multi-Judge Evaluation: Compare GPT-4 vs Claude vs other LLMs as judges
  • πŸ”„ Enhanced meeting data scraping automation
  • πŸ“± Mobile app development
  • πŸ€– Advanced AI categorization improvements
  • πŸ“Š User analytics dashboard

Ready to get started? Follow the Quick Start Guide or run ./scripts/start-dev.sh! πŸš€

About

Demo app for testing & community feedback -- sponsored by Builders & Backers

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published