Skip to content

feat: Add fisherman database migrations and fishing indexer mode #423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: snowmead/feat-fisherman-service-core
Choose a base branch
from

Conversation

snowmead
Copy link
Contributor

@snowmead snowmead commented Jul 23, 2025

TL;DR

Restructured database schema for file associations and added fisherman indexing.

What changed?

  • Separated bsp_file table from the initial file migration into its own migration
  • Created a new msp_file association table to track which MSP is storing which files
  • Added a new "fishing" indexer mode that only tracks file-related events needed for fisherman monitoring
  • Added fisherman service CLI flag and configuration options to the node

Copy link
Contributor Author

snowmead commented Jul 23, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@snowmead snowmead marked this pull request as ready for review July 23, 2025 15:14
@snowmead snowmead force-pushed the snowmead/feat-fisherman-migrations-indexing branch from 5b22334 to e9916c5 Compare July 23, 2025 15:30
@snowmead snowmead force-pushed the snowmead/feat-fisherman-service-core branch from 8eb66e0 to 25c0437 Compare July 28, 2025 12:49
@snowmead snowmead force-pushed the snowmead/feat-fisherman-migrations-indexing branch from 69399eb to d9ef814 Compare July 28, 2025 12:49
snowmead and others added 8 commits July 30, 2025 11:03
- Add MSP-file and BSP-file association tables via dedicated migrations
- Implement fishing mode handler for targeted event filtering
- Add IndexerMode enum with full, lite, and fishing modes
- Update models to track MSP/BSP file associations
- Handle file deletions in BspConfirmStoppedStoring events
- Add conflict handling for duplicate file associations

The fishing mode indexes only essential events needed for fisherman
monitoring: file operations, provider associations, and bucket lifecycle.

🤖 Generated with Claude Code
Co-Authored-By: Claude <[email protected]>
This ensures proper joins between bsp_file and bsp tables in database queries.
- Create configure_and_spawn_indexer_fisherman() function to eliminate code duplication
- Replace 4 identical code blocks across different service initialization functions
- Consolidates indexer mode determination, validation, and service spawning logic

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Add FileDeletionRequested event to fishing handler event matching
- Simplify MspFile::insert to use execute() instead of get_result()
@snowmead snowmead force-pushed the snowmead/feat-fisherman-migrations-indexing branch from 5a9748f to 54b1f27 Compare July 30, 2025 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant