Home

HSBencher Overview

HSBencher is a benchmarking harness written in Haskell and designed to aggregate benchmark data from a distributed execution environment. It can be installed from Hackage.

Using fusion tables and HSBencher conventions without an HSBencher Harness (CSV/Criterion)

HSBencher Harnesses

The goal of HSBencher is to run a bunch of benchmarks, each as a single process and as the only thing running on a machine, and then repeat this process, varying across a parameter space. Properties of these runs (e.g. time elapsed) are recorded and then stored in some kind of database (see "backends" below), where they can be retrieved.

HSBencher works well with using systems like Jenkins to launch benchmark jobs on many machines, while relying on HSBencher to aggregate the benchmark data back to a single place.

Currently, adding an HSBencher script to a project requires writing Haskell code to enumerate the benchmarks and describe their configuration space. In the future we may support using simple text files to describe benchmark configurations (as we did in a much earlier version), but for now Haskell coding is required. Thus, the remainder of this article is split into two sections: to help you either (1) use an existing HSBencher setup (no coding required), or (2) create a new HSBencher setup.

How to use existing HSBencher suites.

• Create your own Fusion table
• Build the benchmark harness executable (e.g. run-benchmarks.hs)
• Run the benchmark executable

How to write run-benchmarks.hs

Backends

• DribbleBackend -- a simple backend for outputting to a local CSV file.
• FusionTableBackend -- Upload data to a Google Fusion Table. Corresponds to module HSBencher.Backend.Fusion, package hsbencher-fusion.
• BigQueryBackend -- This doesn't exist yet, but is planned for Tableau integration.

Build Methods

• BuiltInBuildMethods -- rudimentary cabal, ghc, and make BuildMethods that ship with the core hsbencher package.
• CustomBuildMethod_HowTo --