Add `SCCACHE_BASEDIRS` support #2521

Felixoid · 2025-12-18T21:08:45Z

This is an attempt to fix #35.

The ENV variable SCCACHE_BASEDIRS and configuration parameter basedirs are added.

As well as new tests to validate the behavior.

codecov-commenter · 2025-12-18T21:29:44Z

Codecov Report

❌ Patch coverage is 97.50865% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.31%. Comparing base (951e247) to head (31f9f6b).

Files with missing lines	Patch %	Lines
src/cache/cache.rs	76.00%	18 Missing ⚠️
src/util.rs	96.13%	10 Missing ⚠️
src/compiler/preprocessor_cache.rs	97.47%	3 Missing ⚠️
src/server.rs	78.57%	3 Missing ⚠️
tests/oauth.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2521      +/-   ##
==========================================
+ Coverage   71.17%   72.31%   +1.13%     
==========================================
  Files          64       64              
  Lines       35592    36995    +1403     
==========================================
+ Hits        25334    26752    +1418     
+ Misses      10258    10243      -15

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

src/config.rs

src/util.rs

Felixoid · 2025-12-23T09:24:45Z

I got an idea, that basedirs should be added to the stats command as well

AJIOB · 2025-12-24T09:22:45Z

I got an idea, that basedirs should be added to the stats command as well

I think we can also provide the stat about base dir usage translation:

Number of base dir applied requests
Number of base dir skipped requests

In this case we can see, do we need to provide more/better base dirs or not

docs/Configuration.md

src/cache/cache.rs

src/config.rs

src/util.rs

Felixoid · 2025-12-24T14:58:45Z

Number of base dir applied requests

Number of base dir skipped requests

It's a tricky one. After taking a look, the number of successful substitutions is relatively easy to implement, although the counter should be threaded to the ServerStats.

But how to count the number of skipped directories? What is it? If a base_directory didn't match any of the output at all?

AJIOB · 2025-12-24T19:22:26Z

Number of base dir applied requests

Number of base dir skipped requests

It's a tricky one. After taking a look, the number of successful substitutions is relatively easy to implement, although the counter should be threaded to the ServerStats.

But how to count the number of skipped directories? What is it? If a base_directory didn't match any of the output at all?

No, just to check what number of cache requests has been converted using any of base dirs and what number was kept as an absolute one

AJIOB · 2025-12-24T19:23:42Z

About paths: we can try to normalize paths via https://docs.rs/normpath/latest/normpath/ or something similar instead of doing one more implementation.

Felixoid · 2025-12-29T12:52:06Z

About paths: we can try to normalize paths via https://docs.rs/normpath/latest/normpath/ or something similar instead of doing one more implementation.

It looks like https://doc.rust-lang.org/stable/std/path/fn.absolute.html is what I should use for both cases. It normalizes slashes, but keeps the cases as is.

Although see my comment regarding the preprocessor_output. normalize_path is used not only to needles from basedirs, but to a haystack as well. Both https://doc.rust-lang.org/stable/std/path/fn.absolute.html and https://docs.rs/normpath/latest/normpath/trait.PathExt.html#tymethod.normalize_virtually work on Path-like objects. They don't fit the purpose of the plaintext normalization, as far as I can tell.

Felixoid · 2025-12-30T00:33:50Z

No, just to check what number of cache requests has been converted using any of base dirs and what number was kept as an absolute one

I am afraid it causes changes in too many places, including the signatures of hash_key and other significant components. I want to cut this task from the PR, if you don't mind; it's already quite huge.

src/config.rs

src/util.rs

docs/Configuration.md

README.md

docs/Configuration.md

src/cache/readonly.rs

src/compiler/c.rs

src/util.rs

mathstuf · 2025-12-30T16:01:39Z

src/util.rs

+    for basedir_path in basedirs.iter() {
+        let basedir_str = basedir_path.to_string_lossy();
+        let basedir = basedir_str
+            .trim_end_matches(|c| c == '/' || c == '\\')


\ should not be trimmed on non-Windows.

#2521 (comment)

so, stripping \|/ on windows, / on other OSes, and then adding the trailing /

I'll use https://docs.rs/typed-path/latest/typed_path/ here.

Then, the trailing / will be added explicitly and completely stripped from the preprocessor_output instead of being replaced by ..

The last question is whether ./ should be treated as a special case. Because ./test.c and test.c won't match

In 87151ba, I implemented this logic. The basedir is stripped completely to match /dir/test.c and test.c in the cache.

The ./test.c is still not covered by the code.

src/util.rs

Co-authored-by: whisperity <[email protected]>

Co-authored-by: Alex Overchenko <[email protected]>

Co-authored-by: Ben Boeckel <[email protected]>

Felixoid · 2026-01-11T20:10:41Z

Rebased it on top of the current master

mathstuf

Looks good to me. Just one nit and a possible followup.

Thanks!

mathstuf · 2026-01-13T01:26:41Z

src/compiler/c.rs

+/// If `basedirs` are provided, paths in the preprocessor output will be normalized by
+/// stripping the longest matching basedir prefix. This enables cache hits across different
+/// absolute paths (similar to ccache's CCACHE_BASEDIR).
+#[allow(clippy::too_many_arguments)]


Something for a new PR, but a Builder pattern to deal with these arguments looks like a nice enhancement. The (test) call sites being a zoo of &[] and unlabeled bool literals isn't that nice IMO.

mathstuf · 2026-01-13T01:30:33Z

README.md

+* Multiple developers working with different username paths
+* Working with multiple project checkouts simultaneously
+
+**Note:** Only absolute paths are supported. Relative paths will prevent server from start.


Suggested change

**Note:** Only absolute paths are supported. Relative paths will prevent server from start.

**Note:** Only absolute paths are supported. Relative paths will prevent server from starting.

Felixoid mentioned this pull request Dec 18, 2025

Implement an equivalent to CCACHE_BASEDIR #35

Open

Felixoid force-pushed the add-basedir-configuration branch from bfba6ec to 9eb3241 Compare December 18, 2025 23:45

Felixoid commented Dec 22, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

Felixoid commented Dec 22, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

Felixoid force-pushed the add-basedir-configuration branch from e818064 to d2e6edd Compare December 22, 2025 10:47

AJIOB reviewed Dec 22, 2025

View reviewed changes

src/util.rs Outdated Show resolved Hide resolved

Felixoid force-pushed the add-basedir-configuration branch from ca0a0db to 6f47f36 Compare December 23, 2025 13:57

Felixoid changed the title ~~Add SCCACHE_BASEDIR support~~ Add SCCACHE_BASEDIRS support Dec 23, 2025

Felixoid force-pushed the add-basedir-configuration branch from b5a7d22 to cf0b871 Compare December 23, 2025 23:03

whisperity reviewed Dec 24, 2025

View reviewed changes

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from a97218c to bccc8f4 Compare December 29, 2025 23:13

Felixoid requested review from AJIOB, mathstuf and whisperity December 30, 2025 00:36

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from 0887cea to e42a1f6 Compare December 30, 2025 12:12

AJIOB reviewed Dec 30, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

AJIOB approved these changes Dec 30, 2025

View reviewed changes

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

mathstuf suggested changes Dec 30, 2025

View reviewed changes

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from 6529696 to fae67f8 Compare December 30, 2025 21:50

Felixoid and others added 27 commits January 11, 2026 21:08

Clean out basedir intermediate step, use basedirs

ba4d37d

Cover Windows case for mixed slashes in paths

da5d931

Add Base directories to --show-stats output

fdf0795

Make basedirs match case insensitive on Windows

2e11934

Add trace logs to strip_basedirs, fix remainings

03b869f

Remove outdated test

ec45156

Optimize strip_basedirs for big input

91ef467

Apply suggestions from code review

513a291

Co-authored-by: whisperity <[email protected]>

Move basedir validation to config building

b6b74ff

Add basedirs for remote cache implementations

9f341d3

Address review point for directories separator

1458e71

Simplify strip_basedirs a bit more

e687bd4

Final cleanup of the code for leftovers

29ddde0

Review: less reallocations

316eb66

Apply suggestions from code review

2f118f2

Co-authored-by: Alex Overchenko <[email protected]>

Move normalization to the config building, use bytes for basedirs

d8f6396

Return unnormalized output, fix tests

477fccd

Empty SCCACHE_BASEDIRS overrides, but unset does not

9de64f5

Apply suggestions from code review

6f011d4

Co-authored-by: Ben Boeckel <[email protected]>

Do not trim spaces from paths, safer env overwrite in tests

880a007

Add tests that check uniq basedir per case

15575d4

Return Cow from strip_basedirs

5346fe1

Use typed_path::Utf8TypedPathBuf to normalize directories

d1720fc

Strip basedir completely, normalize once, add integration tests

1d16ad8

Use ascii + utf in normalize_win_path

e14350c

Another review round

b6e9c95

Avoid config conflicts while set_env manipulations

31f9f6b

Felixoid force-pushed the add-basedir-configuration branch from 1d0156e to 31f9f6b Compare January 11, 2026 20:08

mathstuf approved these changes Jan 13, 2026

View reviewed changes

	Note: Only absolute paths are supported. Relative paths will prevent server from start.
	Note: Only absolute paths are supported. Relative paths will prevent server from starting.

Add SCCACHE_BASEDIRS support #2521

Are you sure you want to change the base?

Add SCCACHE_BASEDIRS support #2521

Conversation

Felixoid commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Dec 23, 2025

Uh oh!

AJIOB commented Dec 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Dec 24, 2025

Uh oh!

AJIOB commented Dec 24, 2025

Uh oh!

AJIOB commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Felixoid commented Dec 29, 2025

Uh oh!

Felixoid commented Dec 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mathstuf Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Felixoid Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Felixoid Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Felixoid Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathstuf left a comment

Choose a reason for hiding this comment

Uh oh!

mathstuf Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

mathstuf Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Add `SCCACHE_BASEDIRS` support #2521

Add `SCCACHE_BASEDIRS` support #2521

Felixoid commented Dec 18, 2025 •

edited

Loading

codecov-commenter commented Dec 18, 2025 •

edited

Loading

AJIOB commented Dec 24, 2025 •

edited

Loading

Felixoid commented Jan 11, 2026 •

edited

Loading