Skip to content

Do Not Set CALI_CONFIG if Builtin Caliper #888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

michaelmckinsey1
Copy link
Collaborator

@michaelmckinsey1 michaelmckinsey1 commented Jul 10, 2025

Description

  • Benchmarks with built in Caliper configurations do not want to set CALI_CONFIG, e.g. RAJAPerf and [WIP] Mpi-Pingpong Benchmark #726. This change enables the option of adding a tag builtin-caliper, which the Caliper modifier will check for and add the config as other environment variables SPOT_CONFIG and OTHER_CALI_CONFIG, which the application can reference if need be, but won't be used natively by Caliper.

Adding/modifying a benchmark (docs: Adding a Benchmark)

  • Update repo/raja-perf/application.py

Adding/modifying core functionality, CI, or documentation:

  • Update modifiers/caliper/modifier.py

@michaelmckinsey1 michaelmckinsey1 self-assigned this Jul 10, 2025
@github-actions github-actions bot added feature New feature or request experiment New or modified experiment application labels Jul 10, 2025
@codecov-commenter
Copy link

codecov-commenter commented Jul 10, 2025

Codecov Report

Attention: Patch coverage is 25.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 41.32%. Comparing base (8a80b65) to head (5c84b7b).

Files with missing lines Patch % Lines
modifiers/caliper/modifier.py 25.00% 6 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #888      +/-   ##
===========================================
- Coverage    41.48%   41.32%   -0.17%     
===========================================
  Files           22       22              
  Lines         1509     1515       +6     
  Branches       213      214       +1     
===========================================
  Hits           626      626              
- Misses         838      844       +6     
  Partials        45       45              
Flag Coverage Δ
unittests 41.32% <25.00%> (-0.17%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@michaelmckinsey1
Copy link
Collaborator Author

michaelmckinsey1 commented Jul 11, 2025

Is it possible to read the tags from repo/raja-perf/application.py in modifiers/caliper/modifier.py? I want to read the builtin-caliper tag from specific applications. Currently I am writing a file in place of this functionality.

@michaelmckinsey1 michaelmckinsey1 marked this pull request as ready for review July 15, 2025 21:51
@michaelmckinsey1 michaelmckinsey1 added the ready for review Ready for review label Jul 15, 2025
@michaelmckinsey1 michaelmckinsey1 changed the title [WIP] Do Not Set CALI_CONFIG if Builtin Caliper Do Not Set CALI_CONFIG if Builtin Caliper Jul 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
application experiment New or modified experiment feature New feature or request ready for review Ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants