Skip to content

Conversation

@crystalin
Copy link
Contributor

Summary

  • Implement comprehensive E2E test framework for testing conversation tracking through the actual proxy
  • Add support for dynamic value substitution in test expectations
  • Create mock Claude API server to avoid hitting real API during tests

Details

This PR adds a new packages/e2e package that provides end-to-end testing capabilities for the Claude Nexus Proxy, focusing on conversation tracking and database state validation.

Key Features

  1. JSON-based test case definitions with dynamic value substitution:

    • $new - Expect a new UUID
    • $same - Same as previous request
    • $different - Different from previous request
    • $previous - Reference to previous request ID
    • $main, $branch_*, $compact_* - Branch patterns
  2. Mock Claude API server to return canned responses during testing

  3. Database export script to convert real conversations into test fixtures

  4. Test runner that:

    • Executes requests through the actual proxy
    • Validates database state after each request
    • Supports variable replacement and pattern matching

Architecture

E2E Test Flow:
1. PostgreSQL Container (test database)
2. Mock Claude API Server (returns canned responses)  
3. Proxy Server (configured to use mock API)
4. Test Runner (sends requests and validates database state)

Known Issues

  • Jest has issues with TypeScript ES modules in global setup/teardown
  • Testcontainers can be slow to start PostgreSQL
  • Tests use fixed delays (500ms) for async storage - should be replaced with proper synchronization
  • Some existing subtask detection tests in shared package are failing (pre-existing issue)

Testing

Manual testing approach is documented in packages/e2e/README-E2E.md. Automated Jest setup is included but has module resolution issues.

Future Improvements

  • Replace fixed delays with event-based synchronization
  • Add support for streaming responses
  • Better CI/CD integration
  • More sophisticated mock Claude responses

Test plan

  • TypeScript compilation passes
  • ESLint checks pass
  • Manual E2E test execution (see packages/e2e/manual-test.sh)
  • Review test framework architecture

🤖 Generated with Claude Code

crystalin and others added 2 commits July 3, 2025 07:58
- Add packages/e2e with Jest and Bun test configurations
- Create test runner with dynamic value substitution ($new, $same, $previous, etc.)
- Implement mock Claude API server for testing
- Add TypeScript types for test case definitions
- Create database export script for converting real conversations to fixtures
- Include sample test cases for conversation tracking
- Add global setup/teardown for PostgreSQL and proxy lifecycle
- Document manual testing approach in README

Note: Jest has issues with TypeScript ES modules in global setup.
Recommended to use manual testing approach or Bun test runner for now.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants