[chore][.github] Add Go benchmarks workflow

[mx-psi]

Description

Adds https://codspeed.io/dashboard CI job to visualize Go benchmarks.

Link to tracking issue

Updates #14111

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.25%. Comparing base (fa20a0c) to head (2ba8276).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #14160      +/-   ##
==========================================
- Coverage   92.27%   92.25%   -0.02%     
==========================================
  Files         658      658              
  Lines       41184    41184              
==========================================
- Hits        38001    37993       -8     
- Misses       2179     2184       +5     
- Partials     1004     1007       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕 118 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• lg_log_request/raw_bytes_4850/compressed_bytes_217/compressor_zstd: 11 µs
• lg_log_request/raw_bytes_4850/compressed_bytes_253/compressor_gzip: 36 µs
• lg_log_request/raw_bytes_4850/compressed_bytes_463/compressor_snappy: 1.7 µs
• lg_metric_request/raw_bytes_10991/compressed_bytes_1033/compressor_snappy: 3.7 µs
• lg_metric_request/raw_bytes_10991/compressed_bytes_538/compressor_zstd: 16.9 µs
• lg_metric_request/raw_bytes_10991/compressed_bytes_587/compressor_gzip: 76.6 µs
• lg_trace_request/raw_bytes_7250/compressed_bytes_269/compressor_zstd: 11.1 µs
• lg_trace_request/raw_bytes_7250/compressed_bytes_313/compressor_gzip: 48.4 µs
• lg_trace_request/raw_bytes_7250/compressed_bytes_615/compressor_snappy: 2.3 µs
• md_log_request/raw_bytes_242/compressed_bytes_210/compressor_zstd: 9.5 µs
• md_log_request/raw_bytes_242/compressed_bytes_220/compressor_gzip: 21.8 µs
• md_log_request/raw_bytes_242/compressed_bytes_260/compressor_snappy: 781 ns
• md_metric_request/raw_bytes_376/compressed_bytes_193/compressor_gzip: 24.1 µs
• md_metric_request/raw_bytes_376/compressed_bytes_203/compressor_zstd: 8.2 µs
• md_metric_request/raw_bytes_376/compressed_bytes_215/compressor_snappy: 831 ns
• md_trace_request/raw_bytes_338/compressed_bytes_256/compressor_gzip: 25.8 µs
• md_trace_request/raw_bytes_338/compressed_bytes_265/compressor_zstd: 8.7 µs
• md_trace_request/raw_bytes_338/compressed_bytes_294/compressor_snappy: 862 ns
• sm_log_request/raw_bytes_160/compressed_bytes_159/compressor_zstd: 8.9 µs
• sm_log_request/raw_bytes_160/compressed_bytes_164/compressor_gzip: 19.3 µs
• ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

[GuillaumeLagrange]

Hello @mx-psi !

Thank you for all the feedback, we'll make the fixes ASAP and discuss them in dedicated issues.
One thing that I spot here is that you are calling codspeed run multiple times within the same job, which we unfortunately do not support, and do not plan to support.

The simplest solution to this would be to use a matrix when running the benchmarks, which would also come with the benefit of parallelizing the runs, more info here on how to do this using the action, adapting it to your wrapper should be quite quick.

If you don't want to use a matrix, the second best solution is to do one single codspeed run invocation, with a command that will run all the benches you want to bench.

We'll keep you updated on the other issues for more scoped discussions!

[mx-psi]

@GuillaumeLagrange Thank you for the comment! Thanks also for the prompt replies on the Github issues :) Let me reply to your comments inline

One thing that I spot here is that you are calling codspeed run multiple times within the same job, which we unfortunately do not support, and do not plan to support.

That's helpful to know, thank you! Based on the documentation I assumed that running it multiple times was the only way of supporting multiple modules since I (mistakenly) assumed that only go invocations were supported as an argument to codspeed run. I am trying out running a single codspeed run invocation (see 2ba8276).

This may allow me to simplify things a lot, thanks for the pointer!!

The simplest solution to this would be to use a matrix when running the benchmarks, which would also come with the benefit of parallelizing the runs, more info here on how to do this using the action, adapting it to your wrapper should be quite quick.

I suspect we would hit Github actions limits in terms of the number of workflows that can be run if we go down this route but it's definitely something we can consider for performance/disk usage, even if we only do it partially

[GuillaumeLagrange]

You could also go for an hybrid solution where you shard benches in groups and parallelize to be less extreme in terms of number of runners!

I'll update the codspeed docs in the github actions section to explicitly mention the "one codspeed run invocation` per job limitation.

[not-matthias]

Hey @mx-psi, we've fixed the issue. We're still in the middle of the review and last-minute changes but if you want to try it you can install it from this branch: cod-1694-allow-configuring-cleanup-of-temporary-files

I've tested it on our fork and it worked: https://github.com/AvalancheHQ/opentelemetry-collector/actions/runs/19370101140/job/55423469254 (see the results here: https://codspeed.io/AvalancheHQ/opentelemetry-collector/runs/6917526da8f3100a25b43f5e)

I reduced the benchmark time from 3s (default) to 1s (via the -benchtime parameter) to avoid having the CI run for a long time. You can do this as well at the cost of reducing the number of iterations per benchmark. Alternatively you can also run multiple packages with shards.

We'll merge this next week, let us know if you have further questions. Have a nice weekend!

[mx-psi]

Thank you! We discussed this among approvers and maintainers and we are excited to try this out, I have subscribed to the releases on codspeed-go and I'll give it a try once it is out :)