Fix ZipBinaryProvider unpack reporting + parallelize more #1805
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fixes the incorrect tracking bug introduced here: https://github.com/microsoft/vcpkg-tool/pull/1715/files#r2414987620 * Gets clean_prepare_dir and the last_write_time fixup into the parallelized region Related to microsoft/vcpkg#43309 * Plumbs error handling through to one place * Deletes decompress_in_parallel / cmd_execute_and_capture_output_parallel
vicroms
approved these changes
Oct 9, 2025
vicroms
pushed a commit
to vicroms/vcpkg-tool
that referenced
this pull request
Oct 9, 2025
…1805) * Fixes the incorrect tracking bug introduced here: https://github.com/microsoft/vcpkg-tool/pull/1715/files#r2414987620 * Gets clean_prepare_dir and the last_write_time fixup into the parallelized region Related to microsoft/vcpkg#43309 * Plumbs error handling through to one place * Deletes decompress_in_parallel / cmd_execute_and_capture_output_parallel
BillyONeal
added a commit
to BillyONeal/vcpkg-tool
that referenced
this pull request
Oct 16, 2025
The bug is that https://github.com/microsoft/vcpkg-tool/blob/17ebfccd9b8d8c635407c4a01f06b46d69752100/include/vcpkg/base/parallel-algorithms.h#L78-L81 is missing a return statement, so both the foreground and background thread try to do the thing if and only if there is exactly 1 thing to restore from the binary cache Before microsoft#1805 , clean_prepare_dir was not in the parallel region, so while we did launch a second useless 7z, it didn't tend to cause things to break since the duplicate copies wrote duplicate files. After that change though, one 7z tries to fill the directory while the other thread does clean_prepare_dir. At least one thinks a success happened, so subsequent parts of vcpkg assume that the CONTROL file inside is readable. Example demonstrating that a cache restore needing exactly 1 restore fails while 2 or more succeeds is below. I also changed ZipReadBinaryProvider to print even when not debugging and/or there is an expected success; that would have caught this issue or at least made it easier to explain. ```console PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install rapidcsv --binarysource "clear;x-azblob,https://vcpkgbinarycachewus.blob.core.windows.net/cache,REDACTED,read" Computing installation plan... The following packages will be built and installed: rapidcsv:[email protected] * vcpkg-cmake:x64-windows@2024-04-23 Additional packages (*) will be modified to complete this operation. Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe warning: "C:\Dev\vcpkg-downloads\tools\7zip-25.01-windows\7z.exe" x "C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip" "-oC:\Dev\vcpkg\packages\vcpkg-cmake_x64-windows" -y failed with exit code 2 7-Zip 25.01 (x86) : Copyright (c) 1999-2025 Igor Pavlov : 2025-08-03 Scanning the drive for archives: ERROR: The system cannot find the file specified. C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip System ERROR: The system cannot find the file specified. C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip: note: while extracting this archive Restored 1 package(s) from HTTP servers in 1.3 s. Use --debug to see more details. Installing 1/2 vcpkg-cmake:x64-windows@2024-04-23... no such file or directory PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install zlib rapidcsv --binarysource "clear;x-azblob,REDACTED,read" Computing installation plan... The following packages will be built and installed: rapidcsv:[email protected] * vcpkg-cmake:x64-windows@2024-04-23 zlib:[email protected] Additional packages (*) will be modified to complete this operation. Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Restored 2 package(s) from HTTP servers in 455 ms. Use --debug to see more details. Installing 1/3 vcpkg-cmake:x64-windows@2024-04-23... Elapsed time to handle vcpkg-cmake:x64-windows: 5.29 ms vcpkg-cmake:x64-windows package ABI: 791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64 Installing 2/3 rapidcsv:[email protected]... Building rapidcsv:[email protected]... -- Using cached d99kris-rapidcsv-v8.89.tar.gz -- Extracting source C:/Dev/vcpkg-downloads/d99kris-rapidcsv-v8.89.tar.gz -- Using source at C:/Dev/vcpkg/buildtrees/rapidcsv/src/v8.89-cd345b3fd9.clean -- Configuring x64-windows -- Building x64-windows-dbg -- Building x64-windows-rel -- Installing: C:/Dev/vcpkg/packages/rapidcsv_x64-windows/share/rapidcsv/copyright -- Performing post-build validation Elapsed time to handle rapidcsv:x64-windows: 1.3 s rapidcsv:x64-windows package ABI: a9ddd6a261ed7bb96b94a10d9f1be32ce4acbbd99bdb3d4daeb8d8264d45c637 Installing 3/3 zlib:[email protected]... Elapsed time to handle zlib:x64-windows: 8.29 ms zlib:x64-windows package ABI: 407af7bf6306e8a2c4f56671649adca653b9d1be17bc05ce5f1c66d29e74bb2f Total install time: 1.3 s Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages. Packages installed in this vcpkg installation declare the following licenses: BSD-3-Clause MIT Zlib rapidcsv is header-only and can be used from CMake via: find_path(RAPIDCSV_INCLUDE_DIRS "rapidcsv.h") target_include_directories(main PRIVATE ${RAPIDCSV_INCLUDE_DIRS}) The package zlib is compatible with built-in CMake targets: find_package(ZLIB REQUIRED) target_link_libraries(main PRIVATE ZLIB::ZLIB) All requested installations completed successfully in: 1.3 s PS C:\Dev\vcpkg> ```
BillyONeal
added a commit
that referenced
this pull request
Oct 16, 2025
The bug is that https://github.com/microsoft/vcpkg-tool/blob/17ebfccd9b8d8c635407c4a01f06b46d69752100/include/vcpkg/base/parallel-algorithms.h#L78-L81 is missing a return statement, so both the foreground and background thread try to do the thing if and only if there is exactly 1 thing to restore from the binary cache Before #1805 , clean_prepare_dir was not in the parallel region, so while we did launch a second useless 7z, it didn't tend to cause things to break since the duplicate copies wrote duplicate files. After that change though, one 7z tries to fill the directory while the other thread does clean_prepare_dir. At least one thinks a success happened, so subsequent parts of vcpkg assume that the CONTROL file inside is readable. Example demonstrating that a cache restore needing exactly 1 restore fails while 2 or more succeeds is below. I also changed ZipReadBinaryProvider to print even when not debugging and/or there is an expected success; that would have caught this issue or at least made it easier to explain. ```console PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install rapidcsv --binarysource "clear;x-azblob,https://vcpkgbinarycachewus.blob.core.windows.net/cache,REDACTED,read" Computing installation plan... The following packages will be built and installed: rapidcsv:[email protected] * vcpkg-cmake:x64-windows@2024-04-23 Additional packages (*) will be modified to complete this operation. Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe warning: "C:\Dev\vcpkg-downloads\tools\7zip-25.01-windows\7z.exe" x "C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip" "-oC:\Dev\vcpkg\packages\vcpkg-cmake_x64-windows" -y failed with exit code 2 7-Zip 25.01 (x86) : Copyright (c) 1999-2025 Igor Pavlov : 2025-08-03 Scanning the drive for archives: ERROR: The system cannot find the file specified. C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip System ERROR: The system cannot find the file specified. C:\Dev\vcpkg\buildtrees\vcpkg-cmake_791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64.zip: note: while extracting this archive Restored 1 package(s) from HTTP servers in 1.3 s. Use --debug to see more details. Installing 1/2 vcpkg-cmake:x64-windows@2024-04-23... no such file or directory PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install zlib rapidcsv --binarysource "clear;x-azblob,REDACTED,read" Computing installation plan... The following packages will be built and installed: rapidcsv:[email protected] * vcpkg-cmake:x64-windows@2024-04-23 zlib:[email protected] Additional packages (*) will be modified to complete this operation. Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Restored 2 package(s) from HTTP servers in 455 ms. Use --debug to see more details. Installing 1/3 vcpkg-cmake:x64-windows@2024-04-23... Elapsed time to handle vcpkg-cmake:x64-windows: 5.29 ms vcpkg-cmake:x64-windows package ABI: 791211895bfbb8dac9b79f35ee6d00ea0ee964e62694f558661e8f5a43016a64 Installing 2/3 rapidcsv:[email protected]... Building rapidcsv:[email protected]... -- Using cached d99kris-rapidcsv-v8.89.tar.gz -- Extracting source C:/Dev/vcpkg-downloads/d99kris-rapidcsv-v8.89.tar.gz -- Using source at C:/Dev/vcpkg/buildtrees/rapidcsv/src/v8.89-cd345b3fd9.clean -- Configuring x64-windows -- Building x64-windows-dbg -- Building x64-windows-rel -- Installing: C:/Dev/vcpkg/packages/rapidcsv_x64-windows/share/rapidcsv/copyright -- Performing post-build validation Elapsed time to handle rapidcsv:x64-windows: 1.3 s rapidcsv:x64-windows package ABI: a9ddd6a261ed7bb96b94a10d9f1be32ce4acbbd99bdb3d4daeb8d8264d45c637 Installing 3/3 zlib:[email protected]... Elapsed time to handle zlib:x64-windows: 8.29 ms zlib:x64-windows package ABI: 407af7bf6306e8a2c4f56671649adca653b9d1be17bc05ce5f1c66d29e74bb2f Total install time: 1.3 s Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages. Packages installed in this vcpkg installation declare the following licenses: BSD-3-Clause MIT Zlib rapidcsv is header-only and can be used from CMake via: find_path(RAPIDCSV_INCLUDE_DIRS "rapidcsv.h") target_include_directories(main PRIVATE ${RAPIDCSV_INCLUDE_DIRS}) The package zlib is compatible with built-in CMake targets: find_package(ZLIB REQUIRED) target_link_libraries(main PRIVATE ZLIB::ZLIB) All requested installations completed successfully in: 1.3 s PS C:\Dev\vcpkg> ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to microsoft/vcpkg#43309