remote-feature-flag-controller: Fix flaky test#5730
Conversation
The test for `generateDeterministicRandomNumber` sometimes fails because it relies on the behavior of `uuidv4`, which is non-deterministic, and needs to be more lenient in the range of acceptable return values.
mcmire
added
team-wallet-framework
|
I've ran this test locally a few times and didn't get a failure, so I feel like this should help things. |
github-project-automation
bot
moved this from Needs dev review
to Review finalised - Ready to be merged
in PR review queue
MajorLift
left a comment
MajorLift
left a comment
There was a problem hiding this comment.
Looks like a reasonable fix for flakiness.
| // Each range should have roughly 10% of the values and 40% deviation | ||
| const expectedPerRange = samples / ranges.length; | ||
| const allowedDeviation = expectedPerRange * 0.3; | ||
| const allowedDeviation = expectedPerRange * 0.4; |
There was a problem hiding this comment.
Is 40% a safe range for the test's requirements? It seems like this might relax the test into validating a very-roughly uniform distribution.😅
There was a problem hiding this comment.
That's a great question. I'm not sure 😅 It seems like the range of values is wider than we expect. It also seems that if generateDeterministicRandomNumber is supposed to be deterministic, we don't have to use random UUIDs or user IDs in the test, we can use known values. But I assume that there is a specific reason we wrote the test this way and fixing it further seemed out of scope.
|
LGTM. ! |
github-project-automation
bot
moved this from Review finalised - Ready to be merged
to Merged, Closed or Archived
in PR review queue
5 participants
Explanation
The test for
generateDeterministicRandomNumbersometimes fails because it relies on the behavior ofuuidv4, which is non-deterministic, and needs to be more lenient in the range of acceptable return values.References
Changelog
Checklist