Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qa_volk_32fc_index_* are flaky #700

Open
argilo opened this issue Nov 10, 2023 · 1 comment
Open

qa_volk_32fc_index_* are flaky #700

argilo opened this issue Nov 10, 2023 · 1 comment
Labels

Comments

@argilo
Copy link
Member

argilo commented Nov 10, 2023

While looking for flaky tests with ctest --repeat until-fail:<n>, the qa_volk_32fc_index_* tests turned out to be flaky. Failures are very rare, but can be induced by rounding the test framework's random numbers so that fewer unique values are produced:

diff --git a/lib/qa_utils.cc b/lib/qa_utils.cc
index 4be7b8a..756d1d5 100644
--- a/lib/qa_utils.cc
+++ b/lib/qa_utils.cc
@@ -34,7 +34,7 @@ void random_floats(void* buf, unsigned int n, std::default_random_engine& rnd_en
     T* array = static_cast<T*>(buf);
     std::uniform_real_distribution<T> uniform_dist(T(-1), T(1));
     for (unsigned int i = 0; i < n; i++) {
-        array[i] = uniform_dist(rnd_engine);
+        array[i] = rintf(uniform_dist(rnd_engine) * 10.f) / 10.f;
     }
 }

When multiple indices attain the same maximum value, the generic protokernel selects the first one, but the SIMD protokernels make no attempt to choose the first one (in contrast to 32f_index_*, where the SIMD protokernels are designed to select the first occurrence). However, even if the 32fc_index_* protokernels are refactored to behave the same way as the 32f_index_* protokernels, failures still occur due to floating-point error: the calculated magnitude of a+bj may not equal the calculated magnitude of b+aj. So there may be no hope of guaranteeing that the "first" maximum value is selected, in which case the only solution would be to make the test framework more flexible so that any index whose magnitude is within some tolerance of the maximum is accepted.

@jdemel
Copy link
Contributor

jdemel commented Dec 1, 2023

I'd argue that a specialized test would be great in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants