Merge branch 'tests' into dev

Author: derjogi

README.md

@@ -328,4 +328,8 @@ After running this command, you'll need to either:
 This often helps to run supabase.
 
 Then, to work with it locally, you can access the local instance's info with `supabase status` and use those to manage it (and e.g. set .env vars).
-The **Studio URL** gives you a graphical interface to supabase, and with **Inbucket URL** is a local email smtp server where you can test email signup.
+The **Studio URL** gives you a graphical interface to supabase, and with **Inbucket URL** is a local email smtp server where you can test email signup.
+
+## Testing
+
+For detailed testing instructions, see [TESTING.md](TESTING.md).

TESTING.md

@@ -0,0 +1,190 @@
+# Testing Guide for SimScore API
+
+## Overview
+
+This guide describes the testing approach for SimScore API, covering test organization, setup instructions, and common testing scenarios.
+
+## Test Organization
+
+```
+tests/
+├── api/                   # API endpoint tests
+│   └── v1/
+│       └── routes/
+│           ├── test_ideas.py      # Idea ranking endpoint tests
+│           ├── test_auth.py       # Authentication endpoint tests
+│           └── test_rate_limit.py # Rate limiting tests
+├── integration/           # End-to-end flows
+│   ├── test_auth_basic.py        # Basic authentication flows
+│   ├── test_auth_guest.py        # Guest user flows
+│   └── test_auth_verified.py     # Verified user flows
+└── conftest.py            # Shared test fixtures
+```
+
+## Test Environment Setup
+
+1. Create a `.env` file with test configuration:
+
+```
+# Environment
+ENVIRONMENT=DEV
+
+# API Configuration
+RATE_LIMIT_PER_USER=20/minute
+GLOBAL_RATE_LIMIT=1000/minute
+
+# Test Settings
+SKIP_EMAIL_VERIFICATION=true
+TEST_API_TOKEN=<your-test-token>
+
+# Database (Local Supabase)
+DATABASE_URL=http://127.0.0.1:54321
+DATABASE_KEY=<your-supabase-service-role-key>
+
+# Credits Configuration
+GUEST_DAILY_CREDITS=10
+GUEST_MAX_CREDITS=100
+USER_DAILY_CREDITS=100
+USER_MAX_CREDITS=1000
+```
+
+2. Start the API server in development mode:
+
+```bash
+poetry run uvicorn app.main:app --reload
+```
+
+3. Ensure Supabase is running locally:
+
+```bash
+supabase start
+```
+
+## Running Tests
+
+### API Endpoint Tests
+
+```bash
+# Run all API tests
+poetry run pytest tests/api/
+
+# Run specific API tests
+poetry run pytest tests/api/v1/routes/test_ideas.py
+poetry run pytest tests/api/v1/routes/test_auth.py
+poetry run pytest tests/api/v1/routes/test_rate_limit.py
+```
+
+### Integration Tests
+
+```bash
+# Run all integration tests
+poetry run pytest tests/integration/
+
+# Run specific integration flows
+poetry run pytest tests/integration/test_auth_basic.py
+poetry run pytest tests/integration/test_auth_guest.py
+poetry run pytest tests/integration/test_auth_verified.py
+```
+
+### Running Tests by Marker
+
+```bash
+# Auth-related tests
+poetry run pytest -m verified
+poetry run pytest -m guest
+poetry run pytest -m integration
+
+# Rate limiting tests
+poetry run pytest -m rate_limited
+```
+
+## Test Categories
+
+### Ideas API Tests
+
+Tests the idea ranking endpoint functionality:
+
+- Input validation (minimum/maximum ideas, size limits)
+- Cluster analysis and visualization
+- Advanced features (relationship graphs, cluster naming)
+- Credit system integration
+- Error handling
+
+### Authentication Tests
+
+Tests user management and authentication:
+
+- User registration and login
+- Email verification
+- API key creation and management
+- Rate limiting on auth endpoints
+- User credits and quotas
+
+### Rate Limiting Tests
+
+Verifies API protection against abuse:
+
+- Request limiting based on IP address
+- Appropriate rate limit configuration
+- 429 response handling
+
+## Key Test Fixtures
+
+The `conftest.py` file provides shared fixtures:
+
+- `client`: HTTP client for API requests
+- `test_user`: Dynamically generated test credentials
+- `auth_headers`: Pre-authenticated request headers
+- `mock_ideas`: Sample idea data for testing
+- `mock_credit_service`: Credit system bypass
+- `disable_limiter`: Disables rate limiting during tests
+
+## Troubleshooting
+
+### Rate Limit Errors
+
+If tests fail with 429 status codes:
+
+```bash
+# Option 1: Wait for rate limit reset
+sleep 60
+
+# Option 2: Run with disabled rate limiting
+DISABLE_RATE_LIMITS=true poetry run pytest
+```
+
+### Database Connection Issues
+
+If tests fail to connect to Supabase:
+
+```bash
+# Check if Supabase is running
+supabase status
+
+# Restart Supabase if needed
+supabase stop
+supabase start
+```
+
+### Authentication Failures
+
+For auth test failures:
+
+```bash
+# Ensure email verification is disabled for tests
+SKIP_EMAIL_VERIFICATION=true
+
+# Use a pre-configured test token
+TEST_API_TOKEN=<valid-test-token>
+```
+
+## CI/CD Integration
+
+Tests run automatically on:
+- Pull requests to main branch
+- Nightly builds
+
+The CI pipeline runs tests in a containerized environment with:
+- Isolated test database
+- Test-specific rate limits
+- Email verification disabled

app/api/v1/routes/auth.py

@@ -1,7 +1,9 @@
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, HTTPException, Request
 from pydantic import BaseModel, EmailStr
 from typing import List
 import app.core.security as backend
+from app.core.limiter import limiter  # Import from your limiter module
+from app.core.config import settings
 
 # Request/Response Models
 class UserCredentials(BaseModel):
@@ -30,12 +32,16 @@ class EmailVerification(BaseModel):
 
 router = APIRouter(tags=["auth"])
 
+print(f"Router prefix: {router.prefix}")  # See what prefix is being used
+
 @router.post("/auth/sign_up", response_model=SignupResponse)
-async def signup(credentials: UserCredentials) -> SignupResponse:
+@limiter.limit("5/minute")  # Use slowapi limiter
+async def signup(request: Request, credentials: UserCredentials) -> SignupResponse:
     """
     Register a new user account.
     
     Args:
+        request: FastAPI request object
         credentials: User email and password
         
     Returns:
@@ -51,6 +57,7 @@ async def signup(credentials: UserCredentials) -> SignupResponse:
             email=credentials.email
         )
     except Exception as e:
+        print("Failed: ", str(e))
         if isinstance(e, HTTPException):
             raise e
         raise HTTPException(status_code=400, detail=str(e))
@@ -69,7 +76,7 @@ async def verify_email(verification: EmailVerification) -> MessageResponse:
     Raises:
         HTTPException: 400 if verification fails
     """
-    try:
+    try:   
         await backend.verify_email_code(verification.email, verification.code)
         return MessageResponse(message="Email successfully verified")
     except Exception as e:
@@ -93,13 +100,40 @@ async def create_api_key(credentials: UserCredentials) -> ApiKeyResponse:
         HTTPException: 400 if creation fails, 401 if authentication fails
     """
     try:
+        print("\nCreate API key endpoint hit!")  # See if we reach this endpoint
+        print(f"Creating API key for {credentials.email}")
+        print("\nCreating API key...")
+        print(f"Test environment: {settings.ENVIRONMENT == 'DEV'}; SKIP EMAIL VERIFICATION: {settings.SKIP_EMAIL_VERIFICATION}")
+        log = f"Authenticating user {credentials.email} with password {credentials.password}"
         user = await backend.authenticate_user(credentials.email, credentials.password)
+        log = f"User authenticated: {user}"
+        print(f"User authenticated: {user}")
+        print(f"User metadata: {user.user_metadata}")
+        
+        # Skip verification check in test environment
+        if not (settings.ENVIRONMENT == "TEST" and settings.SKIP_EMAIL_VERIFICATION):
+            log = "Checking if email is verified"
+            # Check if email is verified (only in non-test environment)
+            if not user.user_metadata["email_verified"]:
+                log = "Email not verified"
+                raise HTTPException(
+                    status_code=403,
+                    detail="Email not verified. Please verify your email before creating API keys."
+                )
+        else:
+            log = "Test environment - skipping email verification check"
+            print("Test environment - skipping email verification check")
+            
+        log = "Creating API key"
         api_key = backend.create_api_key(user)
+        log = f"API key created: {api_key}"
+        print(f"API key created: {api_key}")
         return ApiKeyResponse(api_key=api_key)
     except Exception as e:
+        print(f"API key creation error: {str(e)}")
         if isinstance(e, HTTPException):
             raise e
-        raise HTTPException(status_code=400, detail=str(e))
+        raise HTTPException(status_code=400, detail=log)
 
 @router.delete("/auth/revoke_api_key/{key}", response_model=MessageResponse)
 async def delete_api_key(

app/api/v1/routes/ideas.py

@@ -16,7 +16,10 @@
 router = APIRouter(tags=["ideas"])
 
 @router.post("/rank_ideas", response_model=AnalysisResponse)
-@limiter.limit(settings.RATE_LIMIT_PER_USER)
+@limiter.limit(
+    settings.RATE_LIMIT_PER_USER,
+    key_func=lambda request: request.client.host if request.client else "global"
+)
 async def rank_ideas(
     request: Request,
     ideaRequest: IdeaRequest,

app/core/config.py

@@ -1,5 +1,6 @@
 from pydantic_settings import SettingsConfigDict, BaseSettings
 from typing import Dict, TypedDict
+import os
 
 class OperationCost(TypedDict):
     base_cost: float
@@ -50,6 +51,14 @@ class Settings(BaseSettings):
     USER_DAILY_CREDITS: int
     USER_MAX_CREDITS: int
 
+    # Environment
+    ENVIRONMENT: str = "DEV"
+    
+    # Test Configuration
+    SKIP_EMAIL_VERIFICATION: bool = False
+    
+    TEST_API_TOKEN: str = os.getenv("TEST_API_TOKEN", "test-api-token-for-unit-tests")
+    
     model_config = SettingsConfigDict(env_file=".env")
 
 settings = Settings()

app/core/db.py

@@ -4,4 +4,4 @@
 db: Client = create_client(
   settings.DATABASE_URL, 
   settings.DATABASE_KEY
-  )
+)

app/core/limiter.py

@@ -12,14 +12,14 @@
 from app.core.config import settings
 
 def get_identifier(request):
-  """Get unique identifier for rate limiting based on auth status"""
-  if "authorization" in request.headers:
-      return request.headers["authorization"]
-  return get_remote_address(request)
+    """Get unique identifier for rate limiting based on auth status"""
+    if "authorization" in request.headers:
+        return request.headers["authorization"]
+    return get_remote_address(request)
 
 limiter = Limiter(
-  key_func=get_identifier,
-  default_limits=[settings.RATE_LIMIT_PER_USER],
-  strategy="fixed-window",
-  storage_uri="memory://",  # if we have auto-scaling / multiple servers we'd want to change this to a database
+    key_func=get_identifier,
+    default_limits=[settings.RATE_LIMIT_PER_USER],
+    strategy="fixed-window",
+    storage_uri="memory://"  # Use memory storage for testing
 )