Skip to content

Commit 1c827f9

Browse files
committed
Improves API Gateway and event handling features
Adds documentation for agents, improves linting, and enhances API Gateway features with fixes and new functionalities. The changes include: - Introduces documentation for agents and updates the README with new documentation and examples. - Removes pylint quotes, updates pylint rules, fixes issues with file paths, and ensures hyphenated parameters are preserved in dynamic routes. - Modifies exception handling and updates dependencies in `Pipfile.lock`.
1 parent e4b8703 commit 1c827f9

35 files changed

+723
-128
lines changed

.agents/CODEX.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# CODEX.md
2+
3+
Codex, this repository houses **acai_aws**, a DRY toolkit for building AWS Lambda functions that receive API Gateway requests or batched records (DynamoDB, SQS, SNS, S3, Kinesis, MSK, MQ, Firehose, DocumentDB).
4+
5+
## Quick Start
6+
7+
- Install runtime and dev dependencies: `pipenv install --dev`
8+
- Run the unittest discovery suite: `pipenv run test`
9+
- Lint and coverage helpers: `pipenv run lint`, `pipenv run coverage`
10+
- Generate OpenAPI docs: `pipenv run generate`
11+
12+
## Architecture Cheat Sheet
13+
14+
- `acai_aws/apigateway/router.Router` orchestrates requests: before/after hooks, optional auth, OpenAPI-driven validation, and error handling.
15+
- Handlers declare requirements via `@acai_aws.apigateway.requirements.requirements(...)`—timeout, auth, schemas, hooks, and data classes.
16+
- `acai_aws/apigateway/resolver` selects handler modules (directory/pattern/mapping); dynamic segments stay literal (hyphen safe).
17+
- Record processors inherit `acai_aws.base.event.BaseRecordsEvent`; service-specific `Event`/`Record` classes live under `acai_aws/{service}/`.
18+
- Validation flows through `acai_aws.common.validator.Validator`, backed by `jsonschema` or `pydantic` models.
19+
20+
## Testing Notes
21+
22+
- Tests mirror the package structure under `tests/acai_aws/`; mocks reside in `tests/mocks/`.
23+
- Router tests assert full response dicts—double-check expected `path_params` when changing dynamic route handling.
24+
- Coverage target lives in `pipenv run coverage` (pytest + coverage + HTML/JUnit reports).
25+
26+
## Tips While Editing
27+
28+
- Prefer `apply_patch` for manual edits; avoid auto-formatters unless explicitly requested.
29+
- Keep responses JSON-serializable in router flows; `Response.body` defaults to JSON encoding.
30+
- Dynamic route params may include hyphens—do **not** normalize to underscores; rely on segment-level import resolution instead.
31+
- Many records expose `.operation`; filtering via `operations=[...]` occurs in `BaseRecordsEvent`.
32+
33+
## CI Expectations
34+
35+
- Lint score must remain ≥10 (`pipenv run lint`).
36+
- Unit tests run via `pipenv run test` (unittest discovery).
37+
- Publishing reads version from `CIRCLE_TAG`; avoid hardcoding.
38+
39+
Use this sheet to align with the repo’s conventions and keep happy-path programming intact.

.agents/WARP.md

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# WARP.md
2+
3+
This file provides guidance to WARP (warp.dev) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
**acai_aws** is a DRY, configurable, declarative Python library for working with Amazon Web Services Lambdas. It provides:
8+
- Highly configurable API Gateway internal routing
9+
- OpenAPI schema adherence and generation
10+
- Extensible middleware for validation
11+
- Support for multiple AWS event sources (S3, DynamoDB, Kinesis, SNS, SQS, MSK, MQ, DocumentDB, Firehose)
12+
- Happy Path Programming (HPP) philosophy - inputs validated before operations
13+
14+
## Common Commands
15+
16+
### Development Environment
17+
```bash
18+
# Install dependencies (using pipenv)
19+
pipenv install
20+
21+
# Install development dependencies
22+
pipenv install --dev
23+
```
24+
25+
### Testing
26+
```bash
27+
# Run all tests
28+
pipenv run test
29+
30+
# Run with coverage report
31+
pipenv run coverage
32+
```
33+
34+
### Linting
35+
```bash
36+
# Run pylint with custom rules
37+
pipenv run lint
38+
```
39+
40+
### OpenAPI Generation
41+
```bash
42+
# Generate OpenAPI docs from handlers
43+
pipenv run generate
44+
# This generates OpenAPI documentation from handler files in tests/mocks/apigateway/openapi/**/*.py
45+
```
46+
47+
### Running Single Tests
48+
```bash
49+
# Run specific test file
50+
python -m unittest tests.acai_aws.<module>.<test_file>
51+
52+
# Example:
53+
python -m unittest tests.acai_aws.apigateway.test_router
54+
```
55+
56+
## Architecture Overview
57+
58+
### Core Philosophy: Happy Path Programming (HPP)
59+
All inputs are validated before being operated on. The library uses layers of customizable middleware to allow developers to dictate valid inputs without nested conditionals or try/catch blocks.
60+
61+
### Module Structure
62+
63+
The codebase is organized by AWS service event types:
64+
65+
#### Event Handler Pattern
66+
All event handlers follow a common inheritance pattern from `acai_aws.base.event.BaseRecordsEvent`:
67+
68+
```
69+
BaseRecordsEvent (base/event.py)
70+
├── DynamoDBEvent (dynamodb/event.py)
71+
├── S3Event (s3/event.py)
72+
├── KinesisEvent (kinesis/event.py)
73+
├── SNSEvent (sns/event.py)
74+
├── SQSEvent (sqs/event.py)
75+
├── MSKEvent (msk/event.py)
76+
├── MQEvent (mq/event.py)
77+
├── DocumentDBEvent (documentdb/event.py)
78+
└── FirehoseEvent (firehose/event.py)
79+
```
80+
81+
Each event class:
82+
1. Inherits from `BaseRecordsEvent`
83+
2. Sets `self._record_class` to its specific Record class
84+
3. Implements `records` property that returns validated, parsed records
85+
4. Supports optional `data_class` for custom data transformations
86+
87+
#### API Gateway Architecture
88+
89+
The API Gateway module (`acai_aws/apigateway/`) provides a complete routing and validation system:
90+
91+
**Key Components:**
92+
- `Router` - Main routing orchestrator with middleware pipeline
93+
- `Request` - Normalized request object with multi-format body parsing (JSON, XML, form, GraphQL)
94+
- `Response` - Response builder with error tracking
95+
- `Endpoint` - Wrapper around handler functions with requirements metadata
96+
- `Resolver` - Maps incoming requests to handler functions (supports directory and pattern-based routing)
97+
- `Validator` - Request/response validation against OpenAPI specs or inline schemas
98+
99+
**Router Middleware Pipeline:**
100+
1. `before_all` - Pre-processing hook
101+
2. `with_auth` - Authentication/authorization
102+
3. Request validation (OpenAPI or requirements-based)
103+
4. Handler execution
104+
5. Response validation (optional)
105+
6. `after_all` - Post-processing hook
106+
7. Error handling (`on_error`, `on_timeout`)
107+
108+
**Handler Requirements Decorator:**
109+
Handlers use the `@requirements()` decorator to declare validation rules:
110+
111+
```python
112+
from acai_aws.apigateway.requirements import requirements
113+
114+
@requirements(
115+
auth_required=True,
116+
required_headers=['x-api-key'],
117+
required_query=['user_id'],
118+
required_body={'$ref': '#/components/schemas/User'},
119+
timeout=30,
120+
before=pre_hook,
121+
after=post_hook
122+
)
123+
def post(request, response):
124+
response.body = {'result': 'success'}
125+
return response
126+
```
127+
128+
#### Common Utilities
129+
130+
**Validator** (`acai_aws/common/validator.py`):
131+
- Supports JSON Schema (Draft 7) and Pydantic model validation
132+
- Validates request headers, query params, and body
133+
- Validates response schemas
134+
- Used by both API Gateway and event record validation
135+
136+
**Schema** (`acai_aws/common/schema.py`):
137+
- Manages OpenAPI spec loading and reference resolution
138+
- Extracts route specifications for validation
139+
140+
**JSON Helper** (`acai_aws/common/json_helper.py`):
141+
- Handles JSON encoding/decoding with DynamoDB JSON format support
142+
143+
### Testing Structure
144+
145+
Tests mirror the source structure under `tests/acai_aws/`:
146+
- Each module has corresponding test files
147+
- Mocks are in `tests/mocks/` organized by service
148+
- Mock functions demonstrate handler patterns
149+
150+
### OpenAPI Generation
151+
152+
The `apigateway` module includes a CLI tool for generating OpenAPI documentation from handler code:
153+
- Scans handler files for decorated functions
154+
- Extracts requirements metadata
155+
- Generates/updates OpenAPI spec files in JSON/YAML
156+
- Invoked via: `python -m acai_aws.apigateway generate-openapi`
157+
158+
## Code Style
159+
160+
### Linting Rules (from .pylintrc)
161+
- Max line length: 140 characters
162+
- String quotes: single quotes for strings, double quotes for docstrings
163+
- Pylint score must be >= 10 to pass CI
164+
- Private attributes use double underscore: `self.__attribute`
165+
- Protected attributes use single underscore: `self._attribute`
166+
167+
### Naming Conventions
168+
- Functions: `snake_case` (min 3 chars)
169+
- Variables: `snake_case` (min 3 chars)
170+
- Classes: `PascalCase`
171+
- Constants: `UPPER_SNAKE_CASE`
172+
- Methods: `snake_case` (min 3 chars)
173+
174+
## CI/CD
175+
176+
### CircleCI Pipeline
177+
- Python 3.9 environment
178+
- Automated testing with coverage reports
179+
- Pylint execution with score thresholds
180+
- SonarCloud integration for code quality
181+
- PyPI deployment on git tags
182+
183+
### Deployment
184+
Package is deployed to PyPI automatically on tagged releases. Version is read from `CIRCLE_TAG` environment variable.
185+
186+
## Working with Events
187+
188+
### Record-based Event Handlers
189+
When creating handlers for AWS event sources:
190+
191+
1. **Validation Options:**
192+
- `operations`: Filter by operation type (e.g., `['INSERT', 'MODIFY']` for DynamoDB)
193+
- `raise_operation_error`: Raise exception if operation doesn't match
194+
- `required_body`: JSON schema for record body validation
195+
- `raise_body_error`: Raise exception if body validation fails
196+
197+
2. **Data Classes:**
198+
- Set `event.data_class = YourDataClass` to transform records
199+
- Data class receives `record` parameter in constructor
200+
- `event.records` returns list of data class instances
201+
202+
3. **S3-Specific Features:**
203+
- `get_object=True`: Automatically fetch S3 object bodies
204+
- `data_type='json'|'csv'`: Parse object contents
205+
- `delimiter=','`: CSV delimiter (default: comma)
206+
207+
### API Gateway Handlers
208+
209+
1. **File-based Routing:**
210+
- Directory structure maps to routes: `handlers/user/_user_id.py``/user/{user_id}`
211+
- Pattern-based: `_param` becomes `{param}` path parameter
212+
- Method name matches HTTP method: `def get(request, response)`
213+
214+
2. **Request Object:**
215+
- Auto-detects content type and parses body
216+
- Access via: `request.body`, `request.json`, `request.xml`, `request.form`, `request.graphql`
217+
- Path params: `request.path_params['user_id']`
218+
- Query params: `request.query_params['filter']`
219+
220+
3. **Response Object:**
221+
- Set body: `response.body = {'data': 'value'}`
222+
- Set status: `response.code = 201`
223+
- Add errors: `response.set_error('field', 'error message')`
224+
- Check errors: `response.has_errors`
225+
226+
## Dependencies
227+
228+
Core dependencies:
229+
- `boto3` - AWS SDK
230+
- `jsonschema` - JSON Schema validation
231+
- `pydantic` - Data validation with type hints
232+
- `dynamodb_json` - DynamoDB JSON format handling
233+
234+
Development dependencies:
235+
- `pytest` - Testing framework
236+
- `pylint` - Linting
237+
- `moto` - AWS service mocking

.pylintrc

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
ignore=tests,openapi.yml
33
ignore-patterns=test.*?py
44
persistent=no
5-
load-plugins=pylint_quotes
65
unsafe-load-any-extension=no
76

87
[MESSAGES CONTROL]
@@ -84,9 +83,6 @@ max-attributes=7
8483
min-public-methods=0
8584
max-public-methods=20
8685
max-bool-expr=5
87-
string-quote=single
88-
triple-quote=double
89-
docstring-quote=double
9086

9187
[CLASSES]
9288
defining-attr-methods=__init__,__new__,setUp

Pipfile

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ xmltodict = "*"
2525
autopep8 = "*"
2626
moto = "*"
2727
pylint = "*"
28-
pylint-quotes = "*"
2928
pytest = "*"
3029
pytest-html = "*"
3130
pytest-cov = "*"
@@ -34,4 +33,4 @@ pytest-cov = "*"
3433
generate = "python -m acai_aws.apigateway generate-openapi --base=acai_aws/example --handlers=tests/mocks/apigateway/openapi/**/*.py --output=tests/outputs --format=json,yml --delete"
3534
test = "python -m unittest discover"
3635
coverage = "coverage run --source acai_aws/ -m pytest tests/acai_aws --cov=acai_aws --junitxml ./coverage/reports/xunit.xml --cov-report xml:./coverage/reports/coverage.xml --html=./coverage/reports/index.html --self-contained-html --cov-report html:./coverage -p no:warnings -o log_cli=true"
37-
lint = "pylint acai_aws --recursive=y --load-plugins pylint_quotes --fail-under 10 --output-format=text:coverage/lint/report.txt,json:coverage/lint/report.json,parseable,colorized"
36+
lint = "pylint acai_aws --recursive=y --fail-under 10 --output-format=text:coverage/lint/report.txt,json:coverage/lint/report.json,parseable,colorized"

Pipfile.lock

Lines changed: 0 additions & 8 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)