Description
We need to implement a proper Rolling Bloom Filter to replace the current placeholder in the ReliabilityManager. This is needed for efficient message deduplication and acknowledgment tracking.
Tasks:
- Check if there's an appropriate bloom filter library in nim
- If it doesn't exist, implement the RollingBloomFilter type with the following methods:
- add(item: MessageID)
- contains(item: MessageID): bool
- clean() // Remove expired entries
- Implement serialization and deserialization methods for the Bloom filter
- Update the ReliabilityManager to use the new RollingBloomFilter implementation
- Modify wrapOutgoingMessage and unwrapReceivedMessage to properly utilize the Bloom filter
- Implement a periodic cleaning mechanism for the Bloom filter
- Update unit tests to verify the new bloom filter functionality
Acceptance Criteria:
- Rolling Bloom Filter correctly identifies duplicate messages
- Bloom filter maintains a fixed size and periodically removes old entries
- Serialization and deserialization of Bloom filter work correctly for message passing
Additional Considerations:
- Thread-safety when the bloom filter will be accessed concurrently
- Allow the bloom filter to be customized size, false positive rate, time window and clean interval.