Skip to content

Conversation

@BillyONeal
Copy link
Member

The bug is that

if (work_count == 1)
{
work(size_t{});
}
is missing a return statement, so both the foreground and background thread try to do the thing if and only if there is exactly 1 thing to restore from the binary cache

Before #1805 , clean_prepare_dir was not in the parallel region, so while we did launch a second useless 7z, it didn't tend to cause things to break since the duplicate copies wrote duplicate files. After that change though, one 7z tries to fill the directory while the other thread does clean_prepare_dir. At least one thinks a success happened, so subsequent parts of vcpkg assume that the CONTROL file inside is readable.

Example demonstrating that a cache restore needing exactly 1 restore fails while 2 or more succeeds is below.

I also changed ZipReadBinaryProvider to print even when not debugging and/or there is an expected success; that would have caught this issue or at least made it easier to explain.

PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install rapidcsv --binarysource "clear;x-azblob,https://vcpkgbinarycachewus.blob.core.windows.net/cache,REDACTED,read"
Computing installation plan...
The following packages will be built and installed:
    rapidcsv:[email protected]
  * vcpkg-cmake:x64-windows@2024-04-23
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
warning: "C:\Dev\vcpkg-downloads\tools\7zip-25.01-windows\7z.exe" x "C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip" "-oC:\Dev\vcpkg\packages\vcpkg-cmake_x64-windows" -y failed with exit code 2

7-Zip 25.01 (x86) : Copyright (c) 1999-2025 Igor Pavlov : 2025-08-03

Scanning the drive for archives:

ERROR: The system cannot find the file specified.
C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip

System ERROR:
The system cannot find the file specified.

C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip: note: while extracting this archive
Restored 1 package(s) from HTTP servers in 1.3 s. Use --debug to see more details.
Installing 1/2 vcpkg-cmake:x64-windows@2024-04-23...
no such file or directory
PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install zlib rapidcsv --binarysource "clear;x-azblob,REDACTED,read"
Computing installation plan...
The following packages will be built and installed:
    rapidcsv:[email protected]
  * vcpkg-cmake:x64-windows@2024-04-23
    zlib:[email protected]
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
Restored 2 package(s) from HTTP servers in 455 ms. Use --debug to see more details.
Installing 1/3 vcpkg-cmake:x64-windows@2024-04-23...
Elapsed time to handle vcpkg-cmake:x64-windows: 5.29 ms
vcpkg-cmake:x64-windows package ABI: 791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64
Installing 2/3 rapidcsv:[email protected]...
Building rapidcsv:[email protected]...
-- Using cached d99kris-rapidcsv-v8.89.tar.gz
-- Extracting source C:/Dev/vcpkg-downloads/d99kris-rapidcsv-v8.89.tar.gz
-- Using source at C:/Dev/vcpkg/buildtrees/rapidcsv/src/v8.89-cd345b3fd9.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: C:/Dev/vcpkg/packages/rapidcsv_x64-windows/share/rapidcsv/copyright
-- Performing post-build validation
Elapsed time to handle rapidcsv:x64-windows: 1.3 s
rapidcsv:x64-windows package ABI: a9ddd6a261ed7bb96b94a10d9f1be32ce4acbbd99bdb3d4daeb8d8264d45c637
Installing 3/3 zlib:[email protected]...
Elapsed time to handle zlib:x64-windows: 8.29 ms
zlib:x64-windows package ABI: 407af7bf6306e8a2c4f56671649adca653b9d1be17bc05ce5f1c66d29e74bb2f
Total install time: 1.3 s
Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Packages installed in this vcpkg installation declare the following licenses:
BSD-3-Clause
MIT
Zlib
rapidcsv is header-only and can be used from CMake via:

  find_path(RAPIDCSV_INCLUDE_DIRS "rapidcsv.h")
  target_include_directories(main PRIVATE ${RAPIDCSV_INCLUDE_DIRS})

The package zlib is compatible with built-in CMake targets:

    find_package(ZLIB REQUIRED)
    target_link_libraries(main PRIVATE ZLIB::ZLIB)

All requested installations completed successfully in: 1.3 s
PS C:\Dev\vcpkg>

The bug is that https://github.com/microsoft/vcpkg-tool/blob/17ebfccd9b8d8c635407c4a01f06b46d69752100/include/vcpkg/base/parallel-algorithms.h#L78-L81 is missing a return statement, so both the foreground and background thread try to do the thing if and only if there is exactly 1 thing to restore from the binary cache

Before microsoft#1805 , clean_prepare_dir was not in the parallel region, so while we did launch a second useless 7z, it didn't tend to cause things to break since the duplicate copies wrote duplicate files. After that change though, one 7z tries to fill the directory while the other thread does clean_prepare_dir. At least one thinks a success happened, so subsequent parts of vcpkg assume that the CONTROL file inside is readable.

Example demonstrating that a cache restore needing exactly 1 restore fails while 2 or more succeeds is below.

I also changed ZipReadBinaryProvider to print even when not debugging and/or there is an expected success; that would have caught this issue or at least made it easier to explain.

```console
PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install rapidcsv --binarysource "clear;x-azblob,https://vcpkgbinarycachewus.blob.core.windows.net/cache,REDACTED,read"
Computing installation plan...
The following packages will be built and installed:
    rapidcsv:[email protected]
  * vcpkg-cmake:x64-windows@2024-04-23
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
warning: "C:\Dev\vcpkg-downloads\tools\7zip-25.01-windows\7z.exe" x "C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip" "-oC:\Dev\vcpkg\packages\vcpkg-cmake_x64-windows" -y failed with exit code 2

7-Zip 25.01 (x86) : Copyright (c) 1999-2025 Igor Pavlov : 2025-08-03

Scanning the drive for archives:

ERROR: The system cannot find the file specified.
C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip

System ERROR:
The system cannot find the file specified.

C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip: note: while extracting this archive
Restored 1 package(s) from HTTP servers in 1.3 s. Use --debug to see more details.
Installing 1/2 vcpkg-cmake:x64-windows@2024-04-23...
no such file or directory
PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install zlib rapidcsv --binarysource "clear;x-azblob,REDACTED,read"
Computing installation plan...
The following packages will be built and installed:
    rapidcsv:[email protected]
  * vcpkg-cmake:x64-windows@2024-04-23
    zlib:[email protected]
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
Restored 2 package(s) from HTTP servers in 455 ms. Use --debug to see more details.
Installing 1/3 vcpkg-cmake:x64-windows@2024-04-23...
Elapsed time to handle vcpkg-cmake:x64-windows: 5.29 ms
vcpkg-cmake:x64-windows package ABI: 791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64
Installing 2/3 rapidcsv:[email protected]...
Building rapidcsv:[email protected]...
-- Using cached d99kris-rapidcsv-v8.89.tar.gz
-- Extracting source C:/Dev/vcpkg-downloads/d99kris-rapidcsv-v8.89.tar.gz
-- Using source at C:/Dev/vcpkg/buildtrees/rapidcsv/src/v8.89-cd345b3fd9.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: C:/Dev/vcpkg/packages/rapidcsv_x64-windows/share/rapidcsv/copyright
-- Performing post-build validation
Elapsed time to handle rapidcsv:x64-windows: 1.3 s
rapidcsv:x64-windows package ABI: a9ddd6a261ed7bb96b94a10d9f1be32ce4acbbd99bdb3d4daeb8d8264d45c637
Installing 3/3 zlib:[email protected]...
Elapsed time to handle zlib:x64-windows: 8.29 ms
zlib:x64-windows package ABI: 407af7bf6306e8a2c4f56671649adca653b9d1be17bc05ce5f1c66d29e74bb2f
Total install time: 1.3 s
Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Packages installed in this vcpkg installation declare the following licenses:
BSD-3-Clause
MIT
Zlib
rapidcsv is header-only and can be used from CMake via:

  find_path(RAPIDCSV_INCLUDE_DIRS "rapidcsv.h")
  target_include_directories(main PRIVATE ${RAPIDCSV_INCLUDE_DIRS})

The package zlib is compatible with built-in CMake targets:

    find_package(ZLIB REQUIRED)
    target_link_libraries(main PRIVATE ZLIB::ZLIB)

All requested installations completed successfully in: 1.3 s
PS C:\Dev\vcpkg>
```
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a critical bug in the parallel execution code that caused "no such file or directory" errors during binary cache restoration when exactly one package needed to be restored. The issue stemmed from a missing return statement that caused both foreground and background threads to execute the same work, leading to race conditions where one thread would clean a directory while another was trying to extract files into it.

  • Fixed missing return statement in execute_in_parallel for single-item case
  • Updated logging in ZipReadBinaryProvider to always print extraction output regardless of debug mode

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
include/vcpkg/base/parallel-algorithms.h Added missing return statement to prevent dual execution in single-work case
src/vcpkg/binarycaching.cpp Restructured logging to always output extraction details and only show success diagnostics in debug mode

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@BillyONeal BillyONeal merged commit 5a6353c into microsoft:main Oct 16, 2025
7 checks passed
@BillyONeal BillyONeal deleted the debug-temp branch October 16, 2025 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants