This repository was archived by the owner on Aug 13, 2019. It is now read-only.

Attempt to reduce memory footprint of compaction#627

Closed

codesome wants to merge 49 commits intoprometheus-junkyard:masterfrom

codesome:compact-opt

Commits on Jun 11, 2019

Returning series ref in ChunkSeriesSet.At()
Ganesh Vernekar
committed
newCompactionMerger takes []ChunkSeriesSet. Calculating (old series id -> new series id) map.
Ganesh Vernekar
committed
User (old series id -> new series id) map to write new postings
Ganesh Vernekar
committed

Commits on Jun 12, 2019

Get rid of postingMap
Ganesh Vernekar
committed
NumSeries() for BlockReader. Pre-allocate memory for seriesMap maps.
Ganesh Vernekar
committed

Commits on Jun 14, 2019

Set a higher initial value for postingBuf
Ganesh Vernekar
committed
Efficiently iterate over all sorted label-values
Ganesh Vernekar
committed
Hint number of postings writes to the index Writer to pre-allocate memory.
Ganesh Vernekar
committed
Pre-allocate slice for label values in populateBlock
Ganesh Vernekar
committed

Commits on Jun 18, 2019

Re-use bigEndianPostings in populateBlock
Ganesh Vernekar
committed
Reuse string buffer when swapping stringTuples
Ganesh Vernekar
committed
Reuse byte slice for writing chunk meta hash
Ganesh Vernekar
committed
Fix race in writeHash
Ganesh Vernekar
committed
Fix lint errors
Ganesh Vernekar
committed
Reuse slice in ReadOffsetTable. Reduces allocs for OpenBlock.
Ganesh Vernekar
committed
Reuse ListPostings in populateBlock.
Ganesh Vernekar
committed
Check for error before wrapping in blockIndexReader.Series
Ganesh Vernekar
committed
Check for error before wrapping in blockIndexReader.Postings
Ganesh Vernekar
committed
Use already allocated byte buffer in writeHash
Ganesh Vernekar
committed
WriteChunks takes a byte buffer to reuse
Ganesh Vernekar
committed

Commits on Jun 19, 2019

Breakdown generic writeOffsetTable into writeLabelIndexesOffsetTable and writePostingsOffsetTable
Ganesh Vernekar
committed

Commits on Jun 20, 2019

Fix review comments
Ganesh Vernekar
committed
Reset() for Postings interface
Ganesh Vernekar
committed
Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed

Commits on Jun 22, 2019

Fix review comments
Ganesh Vernekar
committed

Commits on Jun 28, 2019

Remove Reset(...) from Postings interface
Ganesh Vernekar
committed
More code comments and fix review comments
Ganesh Vernekar
committed

Commits on Jul 2, 2019

Revert changes to writeOffsetTable
Ganesh Vernekar
committed
Revert checking error before wrapping
Ganesh Vernekar
committed
Revert re-use of 'keys' in ReadOffsetTable
Ganesh Vernekar
committed

Commits on Jul 3, 2019

Revert changes made to stringTuples
Ganesh Vernekar
committed
Revert passing byte buffer to WriteChunks and writeHash
Ganesh Vernekar
committed

Commits on Jul 5, 2019

Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed

Commits on Jul 10, 2019

Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed
hashEntry -> postingsHashEntry after merging upstream
Ganesh Vernekar
committed
writePostings perf improvements
Ganesh Vernekar
committed

Commits on Jul 12, 2019

Fix review comments
Ganesh Vernekar
committed
Add RemappedPostings
Ganesh Vernekar
committed
Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed

Commits on Jul 15, 2019

Fix review comments
Ganesh Vernekar
committed

Commits on Jul 16, 2019

Reset ref in compactionMerger when there are 0 chunks
Ganesh Vernekar
committed

Commits on Jul 23, 2019

compactionMerger doesn't return series with 0 chunks
Ganesh Vernekar
committed
Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed

Commits on Jul 24, 2019

Fix out-of-order series bug
Ganesh Vernekar
committed
Merge remote-tracking branch 'upstream/master' into compact-opt
Ganesh Vernekar
committed
Bug fix in compactionMerger.Next()
Ganesh Vernekar
committed

Commits on Jul 25, 2019

Don't put deleted series in seriesMap
Ganesh Vernekar
committed

Commits on Jul 28, 2019

Better allocation strategy for remapped postings
Ganesh Vernekar
committed

Commits on Aug 2, 2019

Faster way to merge remapped postings
Ganesh Vernekar
committed