Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: Use S2N_FAST_INTEG_TESTS to run pytest in parallel under nix #4368

Merged
merged 5 commits into from
Jan 24, 2024
Merged

feature: Use S2N_FAST_INTEG_TESTS to run pytest in parallel under nix #4368

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
eda4224
feature: Use S2N_FAST_INTEG_TESTS to run pytest in parallel under nix
dougch Jan 17, 2024
df30538
Merge branch 'main' into nix_pytest_auto
dougch Jan 17, 2024
2ae0c25
Merge branch 'main' into nix_pytest_auto
dougch Jan 18, 2024
ddbed1e
make S2N_FAST_INTEG_TESTS on by default
dougch Jan 24, 2024
4e31ef8
Merge branch 'main' into nix_pytest_auto
dougch Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ option(S2N_STACKTRACE "Enables stacktrace functionality in s2n-tls. Note that th
only available on platforms that support execinfo." ON)
option(COVERAGE "Enable profiling collection for code coverage calculation" OFF)
option(S2N_INTEG_TESTS "Enable the integrationv2 tests" OFF)
option(S2N_FAST_INTEG_TESTS "Enable the integrationv2 with more parallelism, only has effect if S2N_INTEG_TESTS=ON" OFF)
option(S2N_FAST_INTEG_TESTS "Enable the integrationv2 with more parallelism, only has effect if S2N_INTEG_TESTS=ON" ON)
option(S2N_INSTALL_S2NC_S2ND "Install the binaries s2nc and s2nd" OFF)
option(TSAN "Enable ThreadSanitizer to test thread safety" OFF)
option(ASAN "Enable AddressSanitizer to test memory safety" OFF)
Expand Down Expand Up @@ -551,6 +551,10 @@ if (BUILD_TESTING)
if (S2N_INTEG_TESTS)
find_package (Python3 COMPONENTS Interpreter Development)
file(GLOB integv2_test_files "${PROJECT_SOURCE_DIR}/tests/integrationv2/test_*.py")
set(N 1)
if (S2N_FAST_INTEG_TESTS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this behind a environment variable? Shouldn't it be on by default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a CMAKE flag that's off by default on https://github.com/aws/s2n-tls/blob/main/CMakeLists.txt#L35 ; since the non-nix integration tests are run with Make, we could flip it to ON, but I'm trying to not to change the behavior too much. Either way, it might be worth adding an else case here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd still like it to be on by default. Since having it off by default just makes it more difficult for people to select the right thing. But at least having it as an option is better than not having it as an option.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flipped to on by default.

set(N auto)
endif()
foreach(test_file_path ${integv2_test_files})
get_filename_component(test_filename ${test_file_path} NAME_WE)
string(REGEX REPLACE "^test_" "integrationv2_" test_target ${test_filename})
Expand All @@ -567,7 +571,7 @@ if (BUILD_TESTING)
add_test(NAME ${test_target}
COMMAND
pytest
-x -n=1 --maxfail=1 --reruns=0 --cache-clear -rpfsq
-x -n=${N} --maxfail=1 --reruns=0 --cache-clear -rpfsq
-o log_cli=true --log-cli-level=DEBUG --provider-version=$ENV{S2N_LIBCRYPTO}
--provider-criterion=off --fips-mode=0 --no-pq=0 ${test_file_path}
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/tests/integrationv2
Expand Down
Loading