fix: batch recovery to avoid 16MB read limit

[sethconvex]

Summary

• Recovery was sending all stale running jobs to a single recover mutation, which could exceed Convex's 16MB read limit when many jobs needed recovery simultaneously (e.g. high maxParallelism + server restart)
• Batch recovery jobs with RECOVERY_BATCH_SIZE = 10 in both the scheduled recover call and the handleRecovery reads inside main
• Keep the 1-minute recovery cadence — remaining old jobs are picked up in subsequent cycles rather than triggering immediate re-runs

Test plan

• All 80 tests pass
• Verify with a high maxParallelism deployment that recovery no longer hits the 16MB limit

🤖 Generated with Claude Code

Summary by CodeRabbit

• Refactor
  • Optimized recovery processing to handle jobs in smaller batches per iteration, reducing resource consumption and improving overall performance during recovery operations.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e1627d4 and b90fbb6.

📒 Files selected for processing (1)

• src/component/loop.ts

---

📝 Walkthrough

Walkthrough

A single file modification to src/component/loop.ts introduces a RECOVERY_BATCH_SIZE constant limiting recovery work to 10 jobs per iteration. The handleRecovery function now processes only a curated batch of old-running jobs instead of scanning all running items, reducing per-iteration processing scope.

Changes

Cohort / File(s)	Summary
Batch Recovery Optimization src/component/loop.ts	Introduces RECOVERY_BATCH_SIZE constant set to 10 and refactors handleRecovery to process only a limited batch of old-running jobs per iteration instead of all running items, with added clarifying comments.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 Hopping through recovery, ten at a time,
No more endless loops in the recovery climb,
Batch by batch we bound, efficient and fleet,
Making recovery loops oh-so-sweet! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: batch recovery to avoid 16MB read limit' directly and clearly summarizes the main change: introducing batching for recovery to prevent exceeding Convex's 16MB read limit.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/recovery-batch-main-loop

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/get-convex/workpool/@convex-dev/workpool@174

commit: b90fbb6