-
-
Notifications
You must be signed in to change notification settings - Fork 788
test(parser): fix UB in test for overlong source #18038
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
test(parser): fix UB in test for overlong source #18038
Conversation
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes undefined behavior (UB) in the overlong_source test that was introduced in #17639. The previous optimization created a fake &str pointing to unallocated memory, which is immediate UB. This PR replaces it with a sound implementation that uses alloc_zeroed to create a 4 GiB allocation efficiently via zero-page mapping.
Changes:
- Replaced the unsafe fake string construction with a proper
ZeroedStringtype - Uses
alloc_zeroedwith page-aligned layout to efficiently allocate 4 GiB of zeroed memory - Implements proper Drop semantics to deallocate the memory
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
CodSpeed Performance ReportMerging this PR will not alter performanceComparing Summary
Footnotes
|
|
@spanishpear Thank you for spotting that this test was so slow and for fixing it. Sorry to go over your work. I'm a bit of a stickler for soundness! Can you possibly help me with something? In description on #17639, you said that this test was taking 7 secs. I was unable to replicate that on my machine - all the parser tests ran in about 700ms on my MacBook Air M3 prior to your PR, 100ms after. So I'm not sure what machine you were seeing the very slow performance on. Can you please advise? If you have time, could you possibly try this branch on that same machine and make sure it's not putting the test back up to 7 secs? If it does, I think we might have to just remove the test entirely. We can't have slow tests, but we should also avoid UB. |
@overlookmotel Thanks for spotting this!! I thought it wasn't UB as long as the underlying bytes weren't addressed, but doing a bit of reading clarified that mistake for me 🙏
I'll take a look! It was on a WSL Ubuntu machine, with much lower specs than an M3 |

#17639 made a nice optimization to the
overlong_sourcetest in parser, which massively speeded it up.Unfortunately, it is unsound. It creates a
&strfrom a range of memory which is not allocated. Creating a&strwhich contains uninitialized bytes is immediate Undefined Behavior, regardless of how the&stris used afterwards.Less academically, if any of the data of the string was accessed, it could try to read from an unallocated memory page, which would be a segfault. That could make this test flaky. The UB could also cause other tests to fail randomly (or pass when they should fail), depending on how compiler exploits the UB.
This PR takes a different approach to speeding up this test. Make an allocation of
MAX_SIZE + 1bytes (4 GiB), but usealloc_zeroedto do it. On most platforms this doesn't actually write zeros across 4 GiB of memory, but just sets all the pages of the allocation to "zero pages" in the page table - which is much faster.This is a bit slower than the unsound approach but in my view still fast enough for our purposes.
Before this PR:
After this PR:
(measured on a Macbook Air M3)
In my view it's worth 300ms to avoid UB.