Skip to content

Commit 9e6c3c9

Browse files
committed
docs: add comprehensive project specification for two-way sync
Add detailed CLAUDE.md specification document covering: - Architecture and data flow - Field mapping (title, description, assignee, status) - Status mapping for GitHub states and OpenProject statuses - Linking strategy using custom field and title prefix - Webhook endpoints with token auth - Startup reconciliation behavior - Configuration and environment variables - Implementation phases and file structure
1 parent cb7cbbf commit 9e6c3c9

File tree

1 file changed

+329
-0
lines changed

1 file changed

+329
-0
lines changed

CLAUDE.md

Lines changed: 329 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,329 @@
1+
# OpenProject-GitHub Two-Way Sync Service
2+
3+
## Project Overview
4+
5+
A Deno-based webhook service that maintains bidirectional synchronization between OpenProject work packages and GitHub repository issues. The service ensures both platforms stay in sync for title, description, assignee, and status fields.
6+
7+
## Architecture
8+
9+
### Service Type
10+
- **Event-driven webhook receiver** (not polling)
11+
- Listens for webhooks from both GitHub and OpenProject
12+
- Performs full reconciliation sync on startup
13+
- Runs as a persistent HTTP server
14+
15+
### Technology Stack
16+
- **Runtime**: Deno (TypeScript)
17+
- **HTTP Server**: Deno's native HTTP server
18+
- **APIs**: GitHub REST API v3, OpenProject API v3
19+
- **Environment**: Docker-ready, configurable via environment variables
20+
21+
## Data Flow
22+
23+
```
24+
GitHub Issue ←→ Sync Service ←→ OpenProject Work Package
25+
↓ ↓
26+
Webhook Webhook
27+
↓ ↓
28+
HTTP Server (this service)
29+
30+
Sync Logic
31+
32+
API Calls to update both sides
33+
```
34+
35+
## Field Mapping
36+
37+
### Synced Fields
38+
39+
| Field | GitHub | OpenProject | Notes |
40+
|-------|--------|-------------|-------|
41+
| Title | `title` | `subject` | GitHub prefixed with `[OP#<id>]` |
42+
| Description | `body` | `description.raw` | Plain text/markdown |
43+
| Assignee | `assignee.login` | `assignee` (user ID) | Mapped via config |
44+
| Status | `state` (open/closed) | `status` | See status mapping below |
45+
46+
### Status Mapping
47+
48+
**GitHub → OpenProject:**
49+
- `open``new` (status ID to be determined from OP API)
50+
- `closed``closed` (status ID to be determined from OP API)
51+
52+
**OpenProject → GitHub:**
53+
- `new`, `in specification`, `specified`, `developed`, `in testing`, `tested`, `test failed`, `on hold``open`
54+
- `closed`, `rejected``closed`
55+
56+
### Assignee Mapping
57+
58+
Configured via environment variable as comma-separated pairs:
59+
60+
```
61+
ASSIGNEE_MAP=github_user1:op_user_id1,github_user2:op_user_id2
62+
```
63+
64+
Example:
65+
```
66+
ASSIGNEE_MAP=johndoe:42,janedoe:123
67+
```
68+
69+
If no mapping exists, the assignee field is not synced for that user.
70+
71+
## Linking Strategy
72+
73+
### OpenProject → GitHub
74+
- Use custom field "GitHub Issue" (integer field) to store GitHub issue number
75+
- Field stores the issue number only (e.g., `123` for issue #123)
76+
77+
### GitHub → OpenProject
78+
- Prefix issue title with `[OP#<id>]` (e.g., `[OP#456] Fix login bug`)
79+
- Parse this prefix to identify linked work packages
80+
81+
### Detecting Links
82+
1. For OP work packages: Check if "GitHub Issue" field is populated
83+
2. For GitHub issues: Check if title starts with `[OP#\d+]` pattern
84+
85+
## Webhook Endpoints
86+
87+
### Authentication
88+
Simple token-based auth using URL path parameter:
89+
90+
```
91+
POST /webhook/:token/github
92+
POST /webhook/:token/openproject
93+
```
94+
95+
The `:token` must match `SECRET_TOKEN` environment variable.
96+
97+
### GitHub Webhook
98+
- **Events to subscribe**: `issues` (opened, edited, closed, reopened, assigned, unassigned)
99+
- **Payload**: Standard GitHub issue webhook payload
100+
101+
### OpenProject Webhook
102+
- **Events to subscribe**: Work package created, updated
103+
- **Payload**: OpenProject webhook payload with work package data
104+
105+
## Startup Behavior
106+
107+
On service startup, perform full reconciliation sync:
108+
109+
1. **Fetch all issues/work packages** for configured repo/project pairs
110+
2. **Identify unlinked items**:
111+
- GitHub issues without `[OP#]` prefix
112+
- OP work packages without "GitHub Issue" field populated
113+
3. **Create corresponding items**:
114+
- For unlinked GitHub issues → Create OP work package
115+
- For unlinked OP work packages → Create GitHub issue
116+
4. **Do NOT attempt to match** existing items (always create new counterparts)
117+
118+
## Conflict Resolution
119+
120+
**Last-write-wins strategy:**
121+
- Compare timestamps from both systems
122+
- Apply the most recent change
123+
- Timestamps to compare:
124+
- GitHub: `issue.updated_at`
125+
- OpenProject: `work_package.updatedAt`
126+
127+
## Configuration
128+
129+
### Environment Variables
130+
131+
```bash
132+
# Authentication tokens
133+
GH_TOKEN=<github_pat> # GitHub Personal Access Token
134+
OP_TOKEN=<openproject_api_key> # OpenProject API Key
135+
SECRET_TOKEN=<webhook_secret> # Webhook URL authentication
136+
137+
# Service URLs
138+
OP_URL=https://op.stoatinternal.com # OpenProject instance URL
139+
140+
# Repository to Project mapping (comma-separated)
141+
REPO_PROJECT_MAP=stoatchat/for-web:8,stoatchat/my-repo:999
142+
143+
# User mapping (optional, comma-separated github_user:op_user_id pairs)
144+
ASSIGNEE_MAP=johndoe:42,janedoe:123
145+
146+
# OpenProject custom field ID for "GitHub Issue" field
147+
OP_GITHUB_ISSUE_FIELD=customField123
148+
```
149+
150+
### Multiple Repository Support
151+
152+
The `REPO_PROJECT_MAP` format:
153+
```
154+
owner/repo:project_id,owner/repo2:project_id2
155+
```
156+
157+
Each mapping creates a bidirectional sync between that GitHub repository and OpenProject project.
158+
159+
## Error Handling
160+
161+
### Strategy
162+
- **Log all errors** to stdout/stderr with context
163+
- **Do not retry** webhook processing (webhooks can be re-delivered)
164+
- **Continue processing** other items if one fails
165+
- **Return HTTP 500** if webhook processing fails entirely
166+
167+
### Error Logging Format
168+
```
169+
[ERROR] [timestamp] [source] Message
170+
Context: {additional context as JSON}
171+
```
172+
173+
## API Interactions
174+
175+
### GitHub API
176+
- **Authentication**: Bearer token (`Authorization: Bearer ${GH_TOKEN}`)
177+
- **Endpoints used**:
178+
- `GET /repos/:owner/:repo/issues` - List issues
179+
- `GET /repos/:owner/:repo/issues/:number` - Get issue
180+
- `POST /repos/:owner/:repo/issues` - Create issue
181+
- `PATCH /repos/:owner/:repo/issues/:number` - Update issue
182+
183+
### OpenProject API
184+
- **Authentication**: Basic auth (`Authorization: Basic ${btoa('apikey:' + OP_TOKEN)}`)
185+
- **Endpoints used**:
186+
- `GET /api/v3/projects/:id/work_packages` - List work packages
187+
- `GET /api/v3/work_packages/:id` - Get work package
188+
- `POST /api/v3/projects/:id/work_packages` - Create work package
189+
- `PATCH /api/v3/work_packages/:id` - Update work package
190+
- `GET /api/v3/statuses` - Get available statuses
191+
- `GET /api/v3/projects/:id/available_assignees` - Get assignees
192+
193+
### Rate Limiting
194+
- Implement basic rate limit handling (retry with exponential backoff if 429 received)
195+
- GitHub: 5000 requests/hour for authenticated requests
196+
- OpenProject: Check response headers for limits
197+
198+
## Implementation Phases
199+
200+
### Phase 1: Core Infrastructure
201+
1. Set up Deno HTTP server with webhook endpoints
202+
2. Environment configuration loading
203+
3. Webhook authentication middleware
204+
4. Basic logging infrastructure
205+
206+
### Phase 2: API Client Layer
207+
1. GitHub API client with authentication
208+
2. OpenProject API client with authentication
209+
3. Type definitions for both APIs
210+
4. Error handling for API calls
211+
212+
### Phase 3: Data Mapping
213+
1. Status mapping logic (with OP status ID lookup on startup)
214+
2. Assignee mapping logic
215+
3. Title formatting (add/remove `[OP#]` prefix)
216+
4. Link detection utilities
217+
218+
### Phase 4: Sync Logic
219+
1. GitHub → OpenProject sync function
220+
2. OpenProject → GitHub sync function
221+
3. Conflict resolution (timestamp comparison)
222+
4. Create missing items on both sides
223+
224+
### Phase 5: Webhook Handlers
225+
1. GitHub webhook parser and handler
226+
2. OpenProject webhook parser and handler
227+
3. Event filtering (only process relevant events)
228+
229+
### Phase 6: Startup Sync
230+
1. Fetch all issues/work packages for configured mappings
231+
2. Identify unlinked items
232+
3. Create corresponding items on both sides
233+
4. Link newly created items
234+
235+
### Phase 7: Testing & Deployment
236+
1. Manual testing with real GitHub/OP instances
237+
2. Docker build and deployment
238+
3. Webhook registration on both platforms
239+
4. Monitoring and logging verification
240+
241+
## File Structure
242+
243+
```
244+
/
245+
├── main.ts # Entry point, HTTP server setup
246+
├── src/
247+
│ ├── config.ts # Environment configuration parsing
248+
│ ├── server.ts # HTTP server and routing
249+
│ ├── middleware/
250+
│ │ └── auth.ts # Webhook authentication
251+
│ ├── clients/
252+
│ │ ├── github.ts # GitHub API client
253+
│ │ └── openproject.ts # OpenProject API client
254+
│ ├── types/
255+
│ │ ├── github.ts # GitHub type definitions
256+
│ │ └── openproject.ts # OpenProject type definitions
257+
│ ├── mappers/
258+
│ │ ├── status.ts # Status mapping logic
259+
│ │ ├── assignee.ts # Assignee mapping logic
260+
│ │ └── link.ts # Link detection/formatting
261+
│ ├── sync/
262+
│ │ ├── github-to-op.ts # GitHub → OP sync
263+
│ │ ├── op-to-github.ts # OP → GitHub sync
264+
│ │ └── reconcile.ts # Startup reconciliation
265+
│ ├── handlers/
266+
│ │ ├── github-webhook.ts # GitHub webhook handler
267+
│ │ └── openproject-webhook.ts # OP webhook handler
268+
│ └── utils/
269+
│ ├── logger.ts # Logging utilities
270+
│ └── errors.ts # Error definitions
271+
├── deno.json # Deno configuration
272+
├── deno.lock # Dependency lock file
273+
├── .env.example # Example environment file
274+
├── Dockerfile # Docker build configuration
275+
├── CLAUDE.md # This file
276+
└── README.md # User documentation
277+
```
278+
279+
## TypeScript Type Definitions
280+
281+
### Core Types
282+
283+
```typescript
284+
// Configuration
285+
interface Config {
286+
githubToken: string;
287+
opToken: string;
288+
opUrl: string;
289+
secretToken: string;
290+
repoProjectMap: Map<string, number>; // "owner/repo" -> project_id
291+
assigneeMap: Map<string, number>; // github_username -> op_user_id
292+
opGithubIssueField: string; // Custom field ID
293+
}
294+
295+
// Link information
296+
interface LinkedPair {
297+
githubRepo: string;
298+
githubIssueNumber: number;
299+
opProjectId: number;
300+
opWorkPackageId: number;
301+
}
302+
303+
// Sync direction
304+
type SyncDirection = 'github-to-op' | 'op-to-github';
305+
```
306+
307+
## Success Criteria
308+
309+
The service is considered complete when:
310+
311+
1. ✅ Webhooks from GitHub update OpenProject work packages
312+
2. ✅ Webhooks from OpenProject update GitHub issues
313+
3. ✅ All specified fields (title, description, assignee, status) sync correctly
314+
4. ✅ Status mapping works according to specification
315+
5. ✅ Assignee mapping works (with graceful handling of missing mappings)
316+
6. ✅ Links are maintained via custom field and title prefix
317+
7. ✅ Startup sync creates missing items on both sides
318+
8. ✅ Conflicts are resolved using last-write-wins
319+
9. ✅ Service runs reliably in Docker
320+
10. ✅ Errors are logged appropriately
321+
322+
## Open Questions / Future Enhancements
323+
324+
- **Deletion handling**: What happens when an issue/work package is deleted?
325+
- **Comment sync**: May be valuable in future
326+
- **Label/tag sync**: Could enhance categorization
327+
- **Bidirectional search**: Web UI to search linked items
328+
- **Metrics/monitoring**: Track sync success rates
329+
- **Database**: Consider storing sync state for better conflict resolution

0 commit comments

Comments
 (0)