Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Rolling Bloom Filter for ReliabilityManager #2

Open
6 of 7 tasks
shash256 opened this issue Oct 18, 2024 · 0 comments · May be fixed by #1
Open
6 of 7 tasks

Implement Rolling Bloom Filter for ReliabilityManager #2

shash256 opened this issue Oct 18, 2024 · 0 comments · May be fixed by #1
Assignees
Labels
enhancement New feature or request

Comments

@shash256
Copy link
Collaborator

shash256 commented Oct 18, 2024

We need to implement a proper Rolling Bloom Filter to replace the current placeholder in the ReliabilityManager. This is needed for efficient message deduplication and acknowledgment tracking.

Tasks:

  • Check if there's an appropriate bloom filter library in nim
  • If it doesn't exist, implement the RollingBloomFilter type with the following methods:
    • add(item: MessageID)
    • contains(item: MessageID): bool
    • clean() // Remove expired entries
  • Implement serialization and deserialization methods for the Bloom filter
  • Update the ReliabilityManager to use the new RollingBloomFilter implementation
    • Modify wrapOutgoingMessage and unwrapReceivedMessage to properly utilize the Bloom filter
  • Implement a periodic cleaning mechanism for the Bloom filter
  • Update unit tests to verify the new bloom filter functionality

Acceptance Criteria:

  • Rolling Bloom Filter correctly identifies duplicate messages
  • Bloom filter maintains a fixed size and periodically removes old entries
  • Serialization and deserialization of Bloom filter work correctly for message passing

Additional Considerations:

  • Thread-safety when the bloom filter will be accessed concurrently
  • Allow the bloom filter to be customized size, false positive rate, time window and clean interval.
@shash256 shash256 self-assigned this Oct 18, 2024
@shash256 shash256 added the enhancement New feature or request label Oct 18, 2024
@shash256 shash256 linked a pull request Oct 21, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant