Update benchmark_step.jl for CUDA benchmarking with useful kernel names
#4055
Conversation
|
|
||
| - group: "Reproducibility infrastructure" | ||
| steps: | ||
|
|
There was a problem hiding this comment.
These changes were made by a YAML auto-formatter in VS Code. Is there a style guide I might be breaking here?
There was a problem hiding this comment.
I'm not sure... this is something I have been wondering as well. I considered following this example, which is used in Buildkite's docs.
|
@dennisYatunin @imreddyTeja thoughts on the timeouts here? Should I increase the limit or disable kernel renaming? |
What is the advantage of using kernel renaming in climaatmos-ci for buildkite steps that don't profile? Is the idea to run the profiler at the end of each simulation? Comparing the buildkite for this PR to the main's last buildkite run shows a ~40% slowdown, but everything after the first step doesn't seem to be affected. I'm not sure if that cost is worth it at the moment. |
I believe that was Dennis's vision, and then we'd process and summarize all of the profiling results together in a later step. |
This reverts commit 427b8db.
|
Alright, so now that CliMA/ClimaCore.jl#2376 is merged, do I need to make a ClimaCore release, or should I simply update the I suppose I could revert any Buildkite changes here and just keep the updates to the benchmarking script. |
| - group: "Benchmarks" | ||
| steps: | ||
|
|
||
| - label: ":computer: Benchmark: CPU baroclinic wave moist" |
There was a problem hiding this comment.
I think this label is incorrect
perf/benchmark_step.jl
Outdated
| redirect_stderr(IOContext(stderr, :stacktrace_types_limited => Ref(false))) | ||
| import ClimaComms | ||
| ClimaComms.@import_required_backends | ||
| import ClimaCore |
There was a problem hiding this comment.
Good catch. It's no longer used. ClimaComms was also imported twice. We really need to figure out how to get linting working!
I think we should make a ClimaCore release. It would be nice if we updated the prettytables compat before releasing. |
This uses CliMA/ClimaCore.jl#2376 to provide more useful CUDA kernel names in benchmarks.
TODO
benchmark.jl, notbenchmark_step.jl? --> we could, but probably not useful for CI yet.