feat(cli): add handler discovery, classification, handler creation, and deployment manifest for flash build #113
+1,614
−23
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This activates the support for deploying Load-balancer -based endpoints. We formerly covered Queue-based only.
- Add comprehensive README.md with complete documentation - Add example GPU worker with @Remote decorator - Add example CPU interface with todo list functions - Add FastAPI main.py entry point - Add .env template with commented values - Add .flashignore and .gitignore - Add requirements.txt with dependencies - Add skeleton.py with conflict detection and template creation
Enables flash init to initialize projects in the current directory with interactive overwrite warnings. Usage: - `flash init` → Initialize in current directory - `flash init .` → Initialize in current directory - `flash init <name>` → Create new directory (existing behavior) Features: - Automatic conflict detection for existing files - Interactive confirmation prompt with file list - `--force` flag to skip prompts - Smart project naming (uses current dir name when in-place) - Conditional "Next steps" (omits cd when in current dir) - Separate success messages for each mode
Fixed critical documentation errors in code examples: - Correct GPU Types - Correct CPU Types
…on README
Address Copilot review feedback on PR 110:
**Quick Start fixes:**
- GPU endpoint: /example/process → /example/
- CPU endpoint: /interface/todo/list → /interface/list
- Fix request model: {"task": "..."} → {"data": "..."}
**API Reference fixes:**
- GPU Worker: /example/process → /example/
- GPU request: {"input_data": {...}} → {"data": "string"}
- CPU list: /interface/todo/list → /interface/list
- CPU add: /interface/todo/add → /interface/add
- CPU delete: DELETE /interface/todo/delete → POST /interface/delete
- Request bodies: {"item": "..."} → {"data": "..."}
- Response format: direct strings → {"result": "string"}
All endpoints now match actual FastAPI router implementation.
Add `if __name__ == "__main__"` test block to CPU interface endpoint to match the pattern in GPU worker endpoint. Features: - Test all three interface functions in parallel using asyncio.gather() - Demonstrates concurrent execution of remote CPU functions - Provides example of running endpoint directly for testing Users can now test CPU interface by running: python workers/interface/endpoint.py
Reverted aggressive lazy-loading patterns that broke IDE reference linking and type checking. Performance is maintained through upstream boto3 lazy-loading in runpod-python. Changes: - Restored direct imports in __init__.py (removed __getattr__ pattern) - Removed get_runpod() lazy-loading function - Fixed ResourceManager to load resources at init - Added Typer decorators to command functions for proper CLI signatures - Improved flash init overwrite behavior with user prompts - Updated tests to handle new initialization pattern Breaking: None - all CLI commands work identically Performance: Maintained with upstream boto3 fix (0.6s cold start) Developer Experience: IDE navigation, autocomplete, and type checking restored
Support both `def` and `async def` function definitions in @Remote decorator. Changes: - Modified AST parser to recognize both FunctionDef and AsyncFunctionDef nodes - Added textwrap.dedent() to handle indented function definitions - Updated client.py docstring with async/sync examples - Added comprehensive test suite for async function support Fixes ValueError when wrapping async functions with @Remote decorator.
Support both `def` and `async def` function definitions in the `@remote` decorator. Changes: - Modified AST parser to recognize both FunctionDef and AsyncFunctionDef nodes - Added textwrap.dedent() to handle indented function definitions - Updated client.py docstring with async/sync examples - Added comprehensive test suite for async function support Fixes ValueError when wrapping async functions with the remote decorator.
…rator' of https://github.com/runpod/tetra-rp into deanq/ae-1469-bug-support-async-and-non-for-remote-decorator
Fixes ValueError when using @Remote decorator with async functions by: - Adding inspect.unwrap() to get original function from wrapper - Supporting ast.AsyncFunctionDef in addition to ast.FunctionDef - Adding early dedenting to handle functions defined in classes/methods This enables proper source extraction for both sync and async decorated functions, resolving the "Could not find function definition" error.
…rator' into deanq/ae-1249-improve-cli
Simplify skeleton template to focus on basic patterns: **GPU Worker (workers/gpu/):** - Simple hello-world response with GPU detection - Returns GPU name, count, and memory via PyTorch - Minimal dependencies (only torch) - Clear demonstration of @Remote decorator **CPU Worker (workers/cpu/):** - Simple hello-world response - No complex data processing - Minimal dependencies (no pandas/numpy) - Clear demonstration of CpuLiveServerless **README Updates:** - Simplified documentation focusing on Tetra patterns - Removed complex ML/data processing examples - Added clear dependency management section - Updated API examples to /hello endpoints **Changes:** - Renamed workers/example/ → workers/gpu/ - Simplified gpu_hello() to return GPU info - Simplified cpu_hello() to basic response - Updated router endpoints from /matrix and /process to /hello - Reduced README from 620 lines to 252 lines (-368 lines) This makes the skeleton more approachable for new users learning the @Remote decorator pattern without the cognitive overhead of understanding PyTorch matrix operations or pandas data analysis.
Update project structure display in flash init command output to show the current skeleton structure: - workers/example/ → workers/gpu/ - workers/interface/ → workers/cpu/ This matches the skeleton template refactoring from commit 590df46.
Simplify flash init command by removing all conda-related functionality: **Removed:** - Conda utility imports - REQUIRED_PACKAGES list - --no-env flag - Conda environment creation and package installation logic - Conditional output based on conda environment status **Simplified:** - Next steps always show: pip install, add API key, flash run - Users manage their own virtual environments (venv, conda, poetry, etc.) - Reduced file from 191 lines to 116 lines (-75 lines) This removes unnecessary complexity and lets users choose their preferred environment management tool.
Update CLI documentation to reflect removal of conda environment management: **flash-init.md:** - Removed --no-env option - Removed conda environment creation documentation - Updated project structure to show gpu/ and cpu/ workers - Simplified next steps to use pip install **README.md:** - Removed conda activate from quick start - Removed --no-env from flash init options - Updated project structure to match skeleton template - Updated test endpoints to /gpu/hello and /cpu/hello Documentation now matches the simplified init command behavior.
Change glob pattern from '*.py' to '**/*.py' in extract_remote_dependencies() to recursively search worker subdirectories. This fixes the build command to work with the new skeleton structure where workers are organized in subdirectories (workers/gpu/, workers/cpu/) instead of directly in the workers/ directory. The build command can now find and extract dependencies from @Remote decorators in workers/gpu/endpoint.py and workers/cpu/endpoint.py.
Add detailed error messages when pip check fails: - Show stderr output from pip --version command - Show exception details if subprocess fails - Add helpful hint to install pip with ensurepip This helps users diagnose why pip detection is failing instead of showing a generic 'pip not found' error.
Detect and use 'uv pip' as fallback when 'python -m pip' is not available. This handles uv-created virtual environments that don't include pip by default. The build command now: 1. Tries python -m pip first 2. Falls back to uv pip if pip not in venv 3. Shows a note when using uv pip 4. Provides helpful install instructions if neither works Fixes build errors in uv environments without pip installed.
…ences - Remove workers/interface/ directory from skeleton template (was supposed to be removed but wasn't) - Fix main.py home endpoint to reference correct paths: /gpu/hello and /cpu/hello (was /gpu/matrix and /cpu/process) - Addresses PR feedback about skeleton complexity
- Change main.py default port from 8000 to 8888 to match flash run command - Ensures consistency between skeleton template and CLI behavior - Documentation already uses port 8888 correctly - Addresses Copilot feedback about port inconsistency
…ld-handler-classification
Rename build and configuration directories from `.tetra` to `.flash` for better branding alignment with Flash CLI. Changes: - Rename TetraPaths to FlashPaths in config.py - Update all directory paths from .tetra to .flash - Update ignore patterns to exclude .flash instead of .tetra - Update deployment.py to use ensure_flash_dir() Note: .tetra_resources.pkl remains unchanged (ResourceManager persistence)
Automatically discover and classify FastAPI handlers as queue-based or load-balancer endpoints for independent serverless deployment. Features: - Discovers FastAPI app and APIRouter routes from main.py and workers/ - Classifies handlers as "queue" (single POST) or "load_balancer" (multiple/GET routes) - Supports explicit type override via ServerlessType.QB/LB - Validates type=QB enforces single POST endpoint requirement - Generates .flash_handlers.json metadata for deployment - Recursively searches workers/ subdirectories including __init__.py files Integration: - Added handler discovery step to flash build command - Created comprehensive test suite (28 tests, 91% coverage) - Updated build.py to use .flash/build/ directory paths - Updated documentation with classification rules and examples Fixes: - Support recursive glob **/*.py to find handlers in subdirectories - Allow __init__.py while skipping private modules (_*.py)
…scovery Remove cross-handler route conflict validation that incorrectly flagged routes with the same path across different handlers (APIRouters). Issue: - gpu_router and cpu_router both have POST /hello - Main app mounts them with different prefixes (/gpu and /cpu) - Actual routes are /gpu/hello and /cpu/hello (no conflict) - Validation was comparing raw paths without prefixes Fix: Each handler (APIRouter) is deployed as an independent serverless endpoint, so route conflicts across handlers are acceptable and expected. The validation now only checks for duplicate handler IDs, not route conflicts. This is correct behavior since: - Each APIRouter becomes a separate deployed endpoint - Routes are isolated per endpoint - Different endpoints can have the same route paths
Temporarily disable build and deploy commands with coming soon message. This release focuses on core init and run functionality, with build and deploy features scheduled for future releases.
…ld-handler-classification
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.