Replication secondary applies the last batch over and over #76
Description
When running in secondary mode OutbackCDX seems to apply the latest write batch over and over even if it has already been applied. I'm not sure this necessarily causes any functional problems but it does mean the index keeps getting updated on disk unnecessarily. The RocksDB log file grows but I assume it will eventually be compacted. It still seems less than ideal though.
How to reproduce:
Run a primary instance:
$ mkdir /tmp/primary
$ java -jar outbackcdx-0.7.0.jar -d /tmp/primary --replication-window 0
Create a collection named 'example' with some record:
$ echo '- 20190101000000 http://example.org/ text/html 200 - - - 1043 333 example.warc.gz' > example.cdx
$ curl --data-binary @example.cdx http://localhost:8080/example
Run a secondary instance:
$ java -jar outbackcdx-0.7.0.jar -d /tmp/secondary -p 8081 --primary http://localhost:8080/example
OutbackCDX http://localhost:8081
Tue Jan 14 17:32:29 KST 2020 ChangePollingThread(http://localhost:8080/example): replicated 1 write batches (1..1) with total length 132 in 0.504s from http://localhost:8080/example/changes?size=10485760&since=0 and our latest sequence number is now 2
Tue Jan 14 17:32:38 KST 2020 ChangePollingThread(http://localhost:8080/example): replicated 1 write batches (1..1) with total length 132 in 0.004s from http://localhost:8080/example/changes?size=10485760&since=1 and our latest sequence number is now 4
Tue Jan 14 17:32:48 KST 2020 ChangePollingThread(http://localhost:8080/example): replicated 1 write batches (1..1) with total length 132 in 0.006s from http://localhost:8080/example/changes?size=10485760&since=1 and our latest sequence number is now 6
Tue Jan 14 17:32:58 KST 2020 ChangePollingThread(http://localhost:8080/example): replicated 1 write batches (1..1) with total length 132 in 0.006s from http://localhost:8080/example/changes?size=10485760&since=1 and our latest sequence number is now 8