Skip to content

How do you save benchmark results? #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FelixBenning opened this issue Oct 5, 2022 · 2 comments · Fixed by #290
Closed

How do you save benchmark results? #289

FelixBenning opened this issue Oct 5, 2022 · 2 comments · Fixed by #290

Comments

@FelixBenning
Copy link
Contributor

FelixBenning commented Oct 5, 2022

https://juliaci.github.io/BenchmarkTools.jl/dev/manual/#Caching-Parameters

  1. Start a Julia session
  2. Execute a benchmark suite using an old version of your package
  3. Save the results somehow (e.g. in a JSON file)

how? The following description is about saving the parameters to a file, but how do I actually save the Benchmark Results?

@FelixBenning
Copy link
Contributor Author

FelixBenning commented Oct 5, 2022

results = run(suite, verbose = true)

BenchmarkTools.save("output.json", median(results))

cf. https://github.com/benchmark-action/github-action-benchmark/blob/master/examples/julia/fib.jl

seems to work and

BenchmarkTools.load("output.json")

also seems to work. So I guess these are just undocumented

@vandre
Copy link

vandre commented May 2, 2023

I found the BenchmarkTools.save method too verbose for my liking. You can also save the benchmark results as follows:

using BenchmarkTools
using JSON

t = @benchmark fib(n) samples=10
data = Dict("mean"=> mean(t).time, "min"=> minimum(t).time, 
"max"=> maximum(t).time, "med"=>median(t).time)
open("output.json", "w") do f  JSON.print(f, data) end

Note: the time units are stored in nanoseconds. See: #307

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants