sanitize_file_name(): Normalize all space characters to a space. #9103

dmsnell · 2025-06-27T21:53:31Z

The sanitize_file_name() function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it.

This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator.

There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions.

github-actions · 2025-06-27T21:53:42Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props dmsnell, jonsurrell, zieladam, desrosj.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

…dPress#9103) Resolves Core-62995 The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions.

github-actions · 2025-06-27T22:11:02Z

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

The Plugin and Theme Directories cannot be accessed within Playground.
All changes will be lost when closing a tab with a Playground instance.
All changes will be lost when refreshing the page.
A fresh instance is created each time the link below is clicked.
Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

sirreal

This seems like an appropriate fix, to simply extend the space replacement to cover the space class. Does this have tests? It would be good to add a case for the known case described in the ticket.

The PHP page on unicode character properties is here. They don't seem to support using the full character property name like JavaScript, which is unfortunate.

I'd consider linking here instead of the data file: https://www.unicode.org/reports/tr44/#General_Category_Values. It seems a bit friendlier for human documentation, but I don't feel strongly.

src/wp-includes/formatting.php

sirreal · 2025-06-30T10:33:13Z

src/wp-includes/formatting.php

+		 * Replace all whitespace characters with a basic space (U+0020).
+		 *
+		 * Characters in the White_Space category are listed with “Zs” in
+		 * their entry in the UnicodeData file maintained at the linked URL.


I think the correct name for this property is "space separator," not "white space".

Suggested change

* Replace all whitespace characters with a basic space (U+0020).

*

* Characters in the White_Space category are listed with “Zs” in

* their entry in the UnicodeData file maintained at the linked URL.

* Replace all whitespace characters with a basic space (U+0020).

*

* Characters in the “Space_Separator” category are listed with “Zs” in

* their entry in the UnicodeData file maintained at the linked URL.

Zl Line separator
Zp Paragraph separator
Zs Space separator

interesting. this document is confusing to me. I think the category of Space_Separator (Zs) is meant to include characters with the binary White_Space property. I’m not entirely sure that all characters with the White_Space property are in the Space_Separator category.

since this is directly referencing Zs in the PCRE pattern, I think your suggestion is best.

I typed with the wording; would appreciate your thoughts

I also find the document confusing. I'm reasonably confident that the Zs category is the correct choice, but I'm less confident about the language we're using to describe it.

I think the modified comment as you've authored it is good 👍 it explains the intent and it links to the relevant documentation for readers to investigate further.

after @adamziel raised his own similar questions I expanded the comment with more links

desrosj · 2025-07-01T00:35:35Z

@dmsnell Would you be able to add some unit tests for this? Changing anything related to formatting without tests makes me pretty nervous.

dmsnell · 2025-07-01T06:32:53Z

Concerning tests I will attempt to add a few but I started a meetup yesterday and don’t have guarantees on my timing.

I welcome anyone wanting to submit some test cases — since this is a PR, feel free to fork my fork and propose a PR whose target branch is this one.

git remote add dmsnell [email protected]:dmsnell/wordpress-develop.git
git fetch dmsnell
git checkout my-tests
git reset --hard dmsnell/filenames/avoid-collation-corruption
# make edits
git push -u origin my-tests

This will create a PR against this branch in my repo, which I can merge into this branch/PR.

…dPress#9103) Resolves Core-62995 The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions.

src/wp-includes/formatting.php

sirreal

This looks like a sufficient, minimal fix for the issue described. Thanks!

dmsnell · 2025-07-01T09:40:21Z

@desrosj what do you think? On this one I would like to have agreement of at least two separate committers.

sirreal · 2025-07-01T09:50:19Z

This does include tests now:

phpunit --testdox --group=16330

Difference with trunk (+3 assertions):

-OK (1 test, 4 assertions)
+OK (1 test, 7 assertions)

If wp-includes/formatting.php is reverted to trunk's version, the tests fail:

_Formatting_Sanitize File Name
 ✘ Replaces spaces
   ┐
   ├ Failed asserting that two strings are identical.
   ┊ ---·Expected
   ┊ +++·Actual
   ┊ @@ @@
   ┊ -'Screenshot-2025-02-19-at-2.17.33-PM.png'
   ┊ +'Screenshot-2025-02-19-at-2.17.33 PM.png'
   │
   ╵ …/tests/phpunit/tests/formatting/sanitizeFileName.php:52
   ┴

…dPress#9103) Resolves Core-62995 The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions.

…ption__suggestions' into filenames/avoid-collation-corruption

desrosj

Thanks for the work on this one, all! I think the approach of expanding to all Space_Separator characters makes sense.

The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions. Developed in #9103 Discussed in https://core.trac.wordpress.org/ticket/62995 Props audrasjb, desrosj, dmsnell, jonsurrell, matt, room34, siliconforks, zieladam. Fixes #62995. git-svn-id: https://develop.svn.wordpress.org/trunk@60399 602fd350-edb4-49c9-b593-d223f7449a82

github-actions · 2025-07-01T15:39:15Z

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 60399
GitHub commit: 77e2afb

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions. Developed in WordPress/wordpress-develop#9103 Discussed in https://core.trac.wordpress.org/ticket/62995 Props audrasjb, desrosj, dmsnell, jonsurrell, matt, room34, siliconforks, zieladam. Fixes #62995. Built from https://develop.svn.wordpress.org/trunk@60399 git-svn-id: http://core.svn.wordpress.org/trunk@59735 1a063a9b-81f0-0310-95a4-ce76da25c4cd

The `sanitize_file_name()` function normalizes the no-break space to a normal space (U+0020) in order to prevent issues saving files with the no-break space in it. This patch expands the replacement to all space characters, since it’s known that macOS stores a NARROW NO-BREAK SPACE (U+202F) in screenshot filenames between the time and the am/pm indicator. There are deeper issues with the way this function works, but this patch resolves a known and common problem without raising any of the deeper refactoring questions. Developed in WordPress/wordpress-develop#9103 Discussed in https://core.trac.wordpress.org/ticket/62995 Props audrasjb, desrosj, dmsnell, jonsurrell, matt, room34, siliconforks, zieladam. Fixes #62995. Built from https://develop.svn.wordpress.org/trunk@60399 git-svn-id: https://core.svn.wordpress.org/trunk@59735 1a063a9b-81f0-0310-95a4-ce76da25c4cd

dmsnell force-pushed the filenames/avoid-collation-corruption branch from ef23a57 to 198fce4 Compare June 27, 2025 21:55

Expand scope of change to appease linter

270964d

sirreal reviewed Jun 30, 2025

View reviewed changes

Add test cases for some characters in the "space separator" category

417119c

sirreal mentioned this pull request Jul 1, 2025

Add tests to filename space separator sanitization dmsnell/wordpress-develop#23

Merged

sirreal reviewed Jul 1, 2025

View reviewed changes

src/wp-includes/formatting.php Show resolved Hide resolved

sirreal approved these changes Jul 1, 2025

View reviewed changes

adamziel approved these changes Jul 1, 2025

View reviewed changes

dmsnell added 5 commits July 1, 2025 17:03

Expand scope of change to appease linter

3b267e9

Merge remote-tracking branch 'sirreal/filenames/avoid-collation-corru…

77436e1

…ption__suggestions' into filenames/avoid-collation-corruption

Update docblock

474216d

Refine the comment, add link to PHP’s PCRE syntax.

5d836bf

dmsnell force-pushed the filenames/avoid-collation-corruption branch from e3ce336 to 5d836bf Compare July 1, 2025 15:03

desrosj approved these changes Jul 1, 2025

View reviewed changes

github-actions bot closed this Jul 1, 2025

dmsnell deleted the filenames/avoid-collation-corruption branch July 1, 2025 15:53

sanitize_file_name(): Normalize all space characters to a space. #9103

sanitize_file_name(): Normalize all space characters to a space. #9103

Uh oh!

Conversation

dmsnell commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 27, 2025

Test using WordPress Playground

Some things to be aware of

Uh oh!

sirreal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sirreal Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

dmsnell Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

dmsnell Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

sirreal Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

dmsnell Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

desrosj commented Jul 1, 2025

Uh oh!

dmsnell commented Jul 1, 2025

Uh oh!

Uh oh!

sirreal left a comment

Choose a reason for hiding this comment

Uh oh!

dmsnell commented Jul 1, 2025

Uh oh!

sirreal commented Jul 1, 2025

Uh oh!

desrosj left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 1, 2025

Uh oh!

Uh oh!

dmsnell commented Jun 27, 2025 •

edited

Loading

github-actions bot commented Jun 27, 2025 •

edited

Loading