You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary:
Sliding window doesn't have a cache for the last item in the previous batch.
This means that if the next batch starts with an offset that was loaded in the previous batch, it will invoke a VERY EXPENSIVE seek backwards.
This expensive seek back was resetting the streams, reloading them (including decompressing them again) and seeking to the right position.
With the new cache, there is no need to seek back anymore.
NOTE: The current cache implementation is built on the assumptions that:
1. Offsets are always increasing
2. We still don't have true sliding window concept (e.g. there are no overlaps).
Reviewed By: sdruzkin
Differential Revision: D77162660
0 commit comments