v2.1.4

What's Changed

This is a patch release consisting of one bugfix concerning the help message showing information regarding telemetry collection.

I also took the occasion to refactor the conanfile.py and improve the instructions to build hictk from source.

Changelog

• [fix]: hictk --help-telemetry by @robomics in #422
• [refactor]: rewrite conanfile.py to be more modular by @robomics in #424
• [docs]: update instruction on how to build hictk from sources by @robomics in #423

Full Changelog: v2.1.3...v2.1.4

v2.1.3

What's Changed

This is a small release consisting of a couple of bugfixes and a few changes to address new warnings issued by GCC 15 and Clang 20.

Changelog

• [fix]: hictk load: respect --drop-unknown-chroms by @robomics in #415
• [perf]: strip hictk binary when compiling in Release by @robomics in #416
• [dependencies]: bump by @robomics and @dependabot in #418 and #419
• [docs]: improve paging of CLI help messages by @robomics in #420

Full Changelog: v2.1.2...v2.1.3

v2.1.2

What's Changed

This is a small release consisting of one bugfix and various improvements to the documentation and error messages.

Changelog

Bugfix

• [bug]: address bug in ToDataFrame transformer when span=full and queries are cis and have unusual shapes by @robomics in #406

Improvements

• [feature]: improve error message generated by hictk balance by @robomics in #401
• [feature]: improve error messages generated when opening .hic files by @robomics in #411

Documentation

• [docs]: update the documentation by @robomics in #412

Various

• [ci]: make build-conan-deps-* workflows more reliable by @robomics in #402
• [deps]: bump by @pre-commit-ci, @dependabot, and @robomics in #326, #405, #407, #410

Full Changelog: v2.1.1...v2.1.2

v2.1.1

What's Changed

This release consists of a single minor bug fix.

• [bug]: fix segfault when fetching genome-wide interactions from empty .hic files by @robomics in #399

Full Changelog: v2.1.0...v2.1.1

v2.1.0

What's Changed

This release contains a mix of new features, performance improvements, and bug fixes.

Highlighted features

Fetching interactions around the matrix diagonal

We introduce a new transformer named DiagonalBand() which given a pair of pixel iterators, returns another pair of iterators that traverse pixels falling within a given matrix diagonal band.
When DiagonalBand() is constructed from iterators coming from hictk's PixelSelectors, pixels are traversed as efficiently as possible by minimizing the number of pixels that are needlessly fetched from the files.

When using hictk::hic::Files, to achieve the best performance possible, be sure to specify the diagonal_band_width parameter when calling the fetch() method.

Improve performance of queries on Cooler files

With this release, we introduce a few heuristics to minimize the amount of unnecessary work that is performed when fetching interactions from Cooler files.
This is especially noticeable when fetching interactions using "tall" queries (e.g. range1=chr1; range2=chrX:10,000,000-11,000,000).
Overall, this resulted in an average performance improvement in the order of 20-40% (with up to 400% improvements in very specific cases, and a few minor regressions when fetching a very small number of interactions from small chromosomes).
See #330 for more details.

Quality of life improvements

• When opening .hic or .mcool files consisting of a single resolution using e.g. hictk::File(), the resolution parameter is no longer mandatory.
• The hictk CLI now supports common units for genomic distances. Example usage: hictk dump myfile.mcool --resolution 5Mbp (5Mb and 5M are equivalent to 5Mbp).
• Better error messages when attempting to open files with an invalid/missing resolution.

Telemetry

With this release, we introduce support for telemetry collection from the hictk CLI.
We collect as little information as possible.
More specifically we do NOT collect information that could be used to:

• identify our users
• identify the machine or environment where hictk is being run
• know which dataset is being processed by hictk
• the parameters used to run hictk

Telemetry collection can be disabled in a few ways, such as defining the HICTK_NO_TELEMETRY env variable before launching hictk.

Furthermore, we added a dedicated page to the documentation at hictk.readthedocs.io with detailed information regarding telemetry collection, processing, and storage.
If you have any questions related to this topic, please start a new discussion on GitHub

Changelog

Changes to library code

• [perf]: improve performance of queries on Cooler files by @robomics in #330
• [feature]: support efficient, lazy fetching of interactions falling within a diagonal band by @robomics in #346
• [enhancement]: update ToDataFrame to use signed integers by @robomics in #358
• [feature]: update transformers to dense/sparse matrices and dataframes to support fetching interactions surrounding the matrix diagonal by @robomics in #356
• [feature]: implement avail_normalizations() on multi-resolution files by @robomics in #362
• [feature]: improve automatic resolution inference by @robomics in #368
• [enhancement]: improve error messages generated by hictk::File() by @robomics in #377 and #357
• [feature]: Support efficient computation of query size by @robomics in #384
• [refactor]: improve layout of internal libs to better support partial builds by @robomics in #397
• [build system]: make overriding git metadata embedding more flexible by @robomics in #352
• [build system]: prefix internal CMake targets with hictk_ to avoid collisions for common names by @robomics in #351

Changes to CLI tools

• [enhancement]: hictk convert: improve handling of files with normalizations not available across all resolutions by @robomics in #365
• [feature]: update CLI to support common units for genomic distances by @robomics in #369
• [perf] hictk validate: greatly improve performance when using --validate-index by @robomics in #333
• [feature]: hictk validate: add CLI option to validate pixels from .[ms]cool files by @robomics in #380
• [feature]: support creating .hic and .cool files with no interactions by @robomics in #387
• [feature]: improve pixel validation when creating .cool/hic files by @robomics in #388
• [feature]: make sure all well-behaved errors result in exit code 1 by @robomics in #370
• [feature]: initial support for collecting telemetry by @robomics in #372
• [fix]: hictk convert: do not hang when processing .cool files with pixels with 0 interactions by @robomics in #379

Updates to CI

• [ci]: refactor by @robomics in #340 and #345
• [ci]: update fuzzy-testing.yml to collect coverage by @robomics in #360
• [ci]: dependabot: reduce update frequency by @robomics in #382
• [ci]: fix coverage reporting by @robomics in #385
• [ci]: fix triggers for clang-tidy GHA workflow by @robomics in #391
• [ci]: use LLD whenever possible by @robomics in #393
• [ci]: fix build-conan-deps* workflows by @robomics in #394
• [fix]: fix build-dockerfile.yml by @robomics in #367
• [fix]: ensure ubuntu-ci.yml uses the generator specified through the job matrix by @robomics in #376
• [pre-commit]: update config and add new hooks by @robomics in #381

Updates to testing

• [fix]: address segfault in unit tests by @robomics in #347
• [testing]: re-organize test datasets by @robomics in #361
• [testing]: improve integration test suite by @robomics in #363 and #371
• [enhancement]: make utils/devel/test_docker_image.sh more generic by @robomics in #375

Various

• [fix]: do not wrap string literals with FMT_STRING when using SPDLOG_ macros by @robomics in #355
• [fix]: fix redirect of stdout to stderr in various scripts by @robomics in #374
• [feature]: use CMake lock files when downloading or generating files by @robomics in #350
• [docs]: Various updates and improvements by @robomics in #353, #390, #392, and #396
• [enhancement]: address clang-tidy warnings by @robomics in #364 and #366
• [deps]: bump by @robomics and @dependabot in #389 and #348

Full Changelog: v2.0.2...v2.1.0

v2.0.2

What's Changed

This is a small maintenance release mostly consisting of bugfixes.

Bugfix

• Add missing include by @robomics in #308
• [bug]: fix enum parsing in hictk CLI code by @robomics in #332
• [bug]: fix mis-tagged unit tests by @robomics in #335
• [bug]: fix handling of Cooler v2 files lacking the "storage-mode" attribute by @robomics in #339

Various

• Improve throughput of float formatting by @robomics in #311
• Address warnings when compiling project with C++23 by @robomics in #310
• hictk: support replaying warnings right before app exit by @robomics in #313
• Bump dependencies @dependabot, @pre-commit-ci and @robomics in in #272, #315, and #317
• Updates to CI by @robomics in #324, #337, and #338
• Update documentation by @robomics in #316 and #320
• [fix]: address warnings raised by CodeQL analysis by @robomics in #319

New Contributors

• @pre-commit-ci made their first contribution in #272

Full Changelog: v2.0.1...v2.0.2

v2.0.1

The main motivation behind this release is to ensure that the up-to-date documentation is properly published on ReadTheDocs.

More details

hictk is relying on the linkcheck plugin to ensure that the documentation does not contain broken or invalid links.
However, when the documentation is being built by the ReadTheDocs CI, some of the links in the docs are not yet valid (but they will be once the up-to-date documentation is published).
linkcheck detected this issue and cancelled the CI job, which is why there is no documentation for hictk v2.0.0 on ReadTheDocs.

PR #299 updates linkcheck's ignore list such that the temporarily invalid links are not checked by linkcheck.

The up-to-date documentation for hictk v2.0.1 is available at: https://hictk.readthedocs.io/en/v2.0.1/

What's Changed

• Update linkcheck ignore list by @robomics in #299
• Remove unused workflow by @robomics in #300
• Implement bins_ptr() accessor for cooler::SingleCellFile() by @robomics in #298
• Prepare for release by @robomics in #301
• Fix typo in the README by @robomics in #303

Full Changelog: v2.0.0...v2.0.1

v2.0.0

What's Changed

• hictk published in Bioinformatics! by @robomics in #194

Changes to library code

• Fix bug when fetching whole-genome interactions as dense or sparse matrix by @robomics in #196, #197
• Expose bin_ptr() accessor wherever it makes sense by @robomics in #199
• Make hictk::common depend on the git-watcher target by @robomics in #233
• Various improvements and bug fixes to code related to fetching interactions as Eigen matrices by @robomics in #227
• Improve transformers::ToDataFrame() by @robomics in #234
• Support merging streams of pixels originating from files of different formats by @robomics in #243
• Fix division by zero when fetching pixels from coolers with bin tables with variable bin size by @robomics in #250
• Various improvements to balancing::Weights by @robomics in #251
• Fix bug when fetching normalized interactions as dense matrices by @robomics in #252
• Minor changes to iterators by @robomics in #253
• Improve support for "square" Cooler files by @robomics in #249
• Rewrite VectorOfAtomicDecimals by @robomics in #255
• Make constructors for To*Matrix transformers more flexible by @robomics in #266
• Improve performance of ToSparseMatrix by reducing memory allocations by @robomics in #277
• Refactor FileStream class by @robomics in #282
• Replace HICTK_UNREACHABLE_CODE macro with hictk::unreachable_code() by @robomics in #285
• Make sure cooler::File::close() does not attempt to finalize files twice by @robomics in #291
• Mark ~TmpDir() noexcept and log errors as warnings using spdlog by @robomics in #292
• Improve logging when failures occur from within class destructors by @robomics in #293
• Add option to exclude "ALL" chromosomes when calling nchroms() on .hic files by @robomics in #294

Changes to CLI tools

• Better handling of coolers with FP counts by @robomics in #201, #202
• Implement hictk metadata command by @robomics in #204, #222
• Improve error message when temporary folder cannot be safely determined by @robomics in #210
• Fix CLI exit code by @robomics in #215
• Add --no-create-weight-link CLI flag where appropriate by @robomics in #223
• Minor improvements to various hictk subcommands by @robomics in #229, #231, #232, #228, #238
• Add option to override count type used to convert .hic files to .[m]cool format by @robomics in #203
• hictk load: support ingesting interactions from pairs/pixel files with interactions overlapping with the lower-triangular matrix by @robomics in #264
• Tweak max active log level based on CMake build type by @robomics in #278
• Fix default log levels when compiling hictk in Debug mode by @robomics in #279
• Ensure no temporary files are written when balancing .hic files with --in-memory by @robomics in #280
• hictk load: support loading compressed interactions by @robomics in #244
• Fix temporary file name collisions in hictk convert *cool -> hic by @robomics in #274

Updates to CI

• Bump GH Actions by @dependabot in #61, #188
• Update .pre-commit-config.yaml by @robomics in #198, #270
• Use prettier to format MD and YAML files by @robomics in #221
• Reduce duration of fuzzy test CI when triggered by PRs by @robomics in #254
• Update cppstd.yml by @robomics in #259
• Disable ccache in fuzzy-testing.yml by @robomics in #260
• Optimize CI workflows by @robomics in #268
• Add workflows to lint project by @robomics in #269
• Use gersemi instead of cmake-format to format CMake files by @robomics in #283
• Bump clang-tidy used by CI by @robomics in #284
• Add macos-15 to CI test matrix by @robomics in #287
• Fix ccache cache eviction logic when using GHA job matrix by @robomics in #288
• Bump compiler used to build Dockerfile to clang-19 by @robomics in #256

Updates to testing

• Add automated tests covering hictk's packaging by @robomics in #246
• Rewrite hictk's fuzzer by @robomics in #226
• Test compatibility with newer C++ standards by @robomics in #258
• Overhaul of the integration test suite by @robomics in #257
• Improve coverage by @robomics in #263
• Increase integration test base timeout by @robomics in #281
• Fix to fuzzer suite by @robomics in #290
• Fix visibility warnings issued when compiling hictk_fuzzer using GCC by @robomics in #276

Various

• Allow dynamically linking to Arrow libs even when building with static linking by @robomics in #200
• Reduce size of hictk's Docker image by @robomics in #209
• Update docs by @robomics in #195, #208
• Refactor by @robomics in #245
• Add missing dep in conanfile.py by @robomics in #265
• Improve compilation times by @robomics in #262
• Fix SPDX headers by @robomics in #267
• Fix Release builds on GCC8 by @robomics in #273
• Bump dependencies by @robomics in #247
• Tweak spdlog by @robomics in #286
• Bump libarchive by @robomics in #289
• Prepare for release by @robomics in #296

Full Changelog: v1.0.0...v2.0.0

v1.0.0

What's Changed

• Fix incorrect overload being called when fetching pixels from cooler files by @robomics in #158
• Properly deal with .hic files with missing normalization vectors by @robomics in #162
• Fix fuzzy tests by @robomics in #165
• hictk balance: add --no-create-weight-link option by @robomics in #167
• hictk balance: honor --in-memory by @robomics in #166
• Add more benchmarks by @robomics in #164
• Throw an error when a non-existing folder is passed to --tmpdir by @robomics in #168
• Switch from conanfile.txt to conanfile.py by @robomics in #169
• Update Docker images used by CI by @robomics in #173
• Refactor code to return interactions as sparse/dense Eigen matrices by @robomics in #176
• Fix typo by @Phlya in #177
• Support fetching interactions as arrow::Table by @robomics in #170
• For the time being, refuse to build on big-endian architectures by @robomics in #181
• Support for fetching interactions from the lower triangle when it makes sense by @robomics in #179
• Drop unnecessary dependency from conanfile.py by @robomics in #184
• Ensure pixel transformers are well-behaved when given an empty range of pixels by @robomics in #182
• Support opening .hic files with only 1 bin per chromosome by @robomics in #180
• Fix typo in conanfile.py by @robomics in #186
• Update docs by @robomics in #183
• Various fixes to the build system by @robomics in #187
• Prepare for release by @robomics in #189

New Contributors

• @Phlya made their first contribution in #177

Full Changelog: v0.0.12...v1.0.0

v0.0.12

What's Changed

• Properly support aggregating .scool with float counts by @robomics in #145
• Install missing targets by @robomics in #146
• Fix segfault when computing the expected value vector by @robomics in #147
• Support chained calls to BinTable.subset() by @robomics in #148
• Drop macOS 11 from CI by @robomics in #149
• Minor updates to the build system by @robomics in #150
• Gracefully deal with files created on Win and read on *NIX by @robomics in #151
• Improve handling of temporary files by @robomics in #152
• Format project by @robomics in #153
• Bugfix: query mirroring by @robomics in #154
• Bump deps by @robomics in #155
• Prepare for release by @robomics in #156

Full Changelog: v0.0.11...v0.0.12