9 tests fail: No such file or directory: 'build/tests/vectors_kem'

[yurivict]

Describe the bug
A clear and concise description of what the bug is.

================================================================================== short test summary info ==================================================================================
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_keygen[ML-KEM-512] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_aft[ML-KEM-768] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_keygen[ML-KEM-768] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_aft[ML-KEM-512] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_val[ML-KEM-512] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_val[ML-KEM-768] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_val[ML-KEM-1024] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_encdec_aft[ML-KEM-1024] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_kem_keygen[ML-KEM-1024] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_kem'
================================================================== 9 failed, 476 passed, 558 skipped in 1182.21s (0:19:42) ==================================================================

To Reproduce
Regular build + test.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: FreeBSD 14.1
• Compiler version used: clang-18
• Python-3.11
• liboqs version: 0.11.0

Additional context
Found in the FreeBSD port.

[bhess]

Thanks for the report @yurivict, I'll try to reproduce it on FreeBSD.

[Sp1l]

Same test for liboqs 0.12.0 on FreeBSD 14.2 amd64

=========================================================== short test summary info ============================================================
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_keygen[ML-DSA-44] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_keygen[ML-DSA-65] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_keygen[ML-DSA-87] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_deterministic[ML-DSA-44] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_deterministic[ML-DSA-65] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_deterministic[ML-DSA-87] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_randomized[ML-DSA-44] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_randomized[ML-DSA-65] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_gen_randomized[ML-DSA-87] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_ver[ML-DSA-44] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_ver[ML-DSA-65] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
FAILED tests/test_acvp_vectors.py::test_acvp_vec_sig_ver[ML-DSA-87] - FileNotFoundError: [Errno 2] No such file or directory: 'build/tests/vectors_sig'
=========================================== 12 failed, 417 passed, 742 skipped in 288.33s (0:04:48) ============================================
FAILED: tests/CMakeFiles/run_tests /wrkdir/usr/ports/security/liboqs/work/.build/tests/CMakeFiles/run_tests

[agc-git]

Same error for liboqs-0.12.0 on NetBSD-10.99.12 aarch64

[Sp1l]

Thanks for the report @yurivict, I'll try to reproduce it on FreeBSD.

Any luck on reproducing?

[bhess]

Thanks for the reminder! I successfully ran the ACVP tests on FreeBSD 14.1 (AArch64) using the latest main branch (or liboqs-0.13.0-rc1):

$ python3.11 tests/test_acvp_vectors.py 
============================================================= test session starts =============================================================
platform freebsd14 -- Python 3.11.10, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/bhe/liboqs
plugins: xdist-3.6.1
collected 255 items                                                                                                                           

tests/test_acvp_vectors.py sssssssssssssssssss...ssssssssssssssssssssssssss...ssssssssssssssssssssssssss...ssssssssss...sssssssssssssss [ 42%]
ssssssssssssssssssssssssssssssssssssss...sssssssssssssssssssssssssssssssssssssssssssssssssssss...ssssssssssssssssssssssssssssssssssssss [ 95%]
ssssssssssss                                                                                                                            [100%]

======================================================= 18 passed, 237 skipped in 2.93s =======================================================

It looks like #2019 may have resolved the issue, but the fix wasn’t included in versions 0.11.0 or 0.12.0. Could you confirm whether the issue is now resolved with the latest version?

[Sp1l]

Looks like that's fixed now, no fails any longer

487 passed, 773 skipped in 352.09s (0:05:52)

[bhess]

Thanks for the confirmation!