In the past, we disabled caching in CI tests (except on MacOs) because it ended up being slower than running without it.
See #9247, #9249, #9319, #9318.
From time to time, we should check if it's convenient to enable the cache.
Windows is the priority because Windows tests are the slowest.
If that goes well, we can look into re-enabling it on Ubuntu too, which runs faster in general.