vcpkg icon indicating copy to clipboard operation
vcpkg copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Versioning fails to checkout ports on shallow vcpkg clones (GitHub Actions default)

Open SamVanheer opened this issue 2 years ago • 14 comments

Describe the bug Certain ports fail to git checkout when the project is built in Github Actions.

Ports known to fail are spdlog and angelscript.

Environment

  • OS: Github Actions windows-latest, ubuntu-latest
  • Compiler: Visual Studio 2022 & G++11

To Reproduce Steps to reproduce the behavior: Repository containing minimal reproduction: https://github.com/SamVanheer/vcpkg-checkout-bug

When this repository's workflow runs it will fail to checkout spdlog.

Expected behavior All dependencies are correctly checked out by Git during the vcpkg install step.

Failure logs See the Github Actions workflow run for the logs: https://github.com/SamVanheer/vcpkg-checkout-bug/actions/runs/2531020466

Additional context The original project where this problem manifested had the ports as submodules, but since the problem also occurs in an independent repository with no previous Git history i doubt this has anything to do with the problem.

SamVanheer avatar Jun 20 '22 19:06 SamVanheer

Thanks for posting this issue. Could you please provide the error log: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\build\vcpkg-manifest-install.log? I can install spdlog succeed locally by vcpkg.json which belong to your repro project. We cann'y get detail reason by the following error message.

Error: while checking out port spdlog with git tree 1ac7[9](https://github.com/SamVanheer/vcpkg-checkout-bug/runs/6972842645?check_suite_focus=true#step:3:10)5913d8824[11](https://github.com/SamVanheer/vcpkg-checkout-bug/runs/6972842645?check_suite_focus=true#step:3:12)71b45d796d3fe5dd38519d5a
Error: Failed to tar port directory
error: tar failed with exit code: (128).
fatal: not a tree object: 1ac795913d88241171b45d796d3fe5dd38519d5a

My install log:

E:\test25349\build>cmake -G "Ninja" "-DCMAKE_TOOLCHAIN_FILE=E:/Lily/0608/scripts/buildsystems/vcpkg.cmake" ..
-- Running vcpkg install
Detecting compiler hash for triplet "x64-windows"...
The following packages will be built and installed:
  * fmt[core]:x64-windows -> 8.1.1#1 -- E:\Lily\0608\buildtrees\versioning_\versions\fmt\602d9743b7957c9e82a06d0e81d58637c6df5222
    spdlog[core]:x64-windows -> 1.10.0#1 -- E:\Lily\0608\buildtrees\versioning_\versions\spdlog\3256ea88cc375fda2f977a2eb18435e23d498572
  * vcpkg-cmake[core]:x64-windows -> 2022-05-10#1 -- E:\Lily\0608\buildtrees\versioning_\versions\vcpkg-cmake\8b07d914c90cf8f611973318c85d3af13201e3f9
  * vcpkg-cmake-config[core]:x64-windows -> 2022-02-06#1 -- E:\Lily\0608\buildtrees\versioning_\versions\vcpkg-cmake-config\8d54cc4f487d51b655abec5f9c9c3f86ca83311f
Additional packages (*) will be modified to complete this operation.
Restored 1 packages from C:\Users\v-lilywang\AppData\Local\vcpkg\archives in 165.7 ms. Use --debug to see more details.
Installing 1/4 vcpkg-cmake-config:x64-windows...
Elapsed time to handle vcpkg-cmake-config:x64-windows: 8.105 ms
Installing 2/4 vcpkg-cmake:x64-windows...
Building vcpkg-cmake[core]:x64-windows...
-- Installing port from location: E:\Lily\0608\buildtrees\versioning_\versions\vcpkg-cmake\8b07d914c90cf8f611973318c85d3af13201e3f9
-- Installing: E:/Lily/0608/packages/vcpkg-cmake_x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake
-- Installing: E:/Lily/0608/packages/vcpkg-cmake_x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake
-- Installing: E:/Lily/0608/packages/vcpkg-cmake_x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake
-- Installing: E:/Lily/0608/packages/vcpkg-cmake_x64-windows/share/vcpkg-cmake/vcpkg-port-config.cmake
-- Installing: E:/Lily/0608/packages/vcpkg-cmake_x64-windows/share/vcpkg-cmake/copyright
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\v-lilywang\AppData\Local\vcpkg\archives\c5\c57d012c163a8b24d0ec2b7e9567507720a6dea162c60f2404ea18d0cbb17bfc.zip
Elapsed time to handle vcpkg-cmake:x64-windows: 240.6 ms
Installing 3/4 fmt:x64-windows...
Building fmt[core]:x64-windows...
-- Installing port from location: E:\Lily\0608\buildtrees\versioning_\versions\fmt\602d9743b7957c9e82a06d0e81d58637c6df5222
-- Using cached fmtlib-fmt-8.1.1.tar.gz.
-- Cleaning sources at E:/Lily/0608/buildtrees/fmt/src/8.1.1-11f8359597.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source E:/Lily/0608/downloads/fmtlib-fmt-8.1.1.tar.gz
-- Applying patch fix-write-batch.patch
-- Applying patch fix-invalid-command.patch
-- Using source at E:/Lily/0608/buildtrees/fmt/src/8.1.1-11f8359597.clean
-- Found external ninja('1.10.2').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: E:/Lily/0608/packages/fmt_x64-windows/share/fmt/copyright
-- Fixing pkgconfig file: E:/Lily/0608/packages/fmt_x64-windows/lib/pkgconfig/fmt.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at E:/Lily/0608/downloads/tools/msys2/9a1ec3f33446b195
-- Fixing pkgconfig file: E:/Lily/0608/packages/fmt_x64-windows/debug/lib/pkgconfig/fmt.pc
-- Installing: E:/Lily/0608/packages/fmt_x64-windows/share/fmt/usage
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\v-lilywang\AppData\Local\vcpkg\archives\ac\ac2c244efebeb101919ee97bb5085399d284ba4d62affc8a8f6bd14a4453a7f2.zip
Elapsed time to handle fmt:x64-windows: 14.13 s
Installing 4/4 spdlog:x64-windows...
Building spdlog[core]:x64-windows...
-- Installing port from location: E:\Lily\0608\buildtrees\versioning_\versions\spdlog\3256ea88cc375fda2f977a2eb18435e23d498572
-- Using cached gabime-spdlog-v1.10.0.tar.gz.
-- Cleaning sources at E:/Lily/0608/buildtrees/spdlog/src/v1.10.0-3cbe543323.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source E:/Lily/0608/downloads/gabime-spdlog-v1.10.0.tar.gz
-- Using source at E:/Lily/0608/buildtrees/spdlog/src/v1.10.0-3cbe543323.clean
-- Found external ninja('1.10.2').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig file: E:/Lily/0608/packages/spdlog_x64-windows/lib/pkgconfig/spdlog.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at E:/Lily/0608/downloads/tools/msys2/9a1ec3f33446b195
-- Fixing pkgconfig file: E:/Lily/0608/packages/spdlog_x64-windows/debug/lib/pkgconfig/spdlog.pc
-- Installing: E:/Lily/0608/packages/spdlog_x64-windows/share/spdlog/copyright
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\v-lilywang\AppData\Local\vcpkg\archives\34\34eef501200bad34430b3bdfe7ffd252e74218882d8d3787f125f883fe9331d1.zip
Elapsed time to handle spdlog:x64-windows: 14.47 s

Total elapsed time: 38.81 s

spdlog provides CMake targets:
    # this is heuristically generated, and may not be correct
    find_package(spdlog CONFIG REQUIRED)
    target_link_libraries(main PRIVATE spdlog::spdlog spdlog::spdlog_header_only)

-- Running vcpkg install - done
CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "E:/test25349/build/CMakeFiles/CMakeOutput.log".

LilyWangLL avatar Jun 21 '22 07:06 LilyWangLL

I've added some logic to print the log file: https://github.com/SamVanheer/halflife-unified-sdk/runs/6969638872?check_suite_focus=true

But all it seems to contain is this:

Error: while checking out port spdlog with git tree 1ac795913d88241171b45d796d3fe5dd38519d5a
Error: Failed to tar port directory
error: tar failed with exit code: (128).
fatal: not a tree object: 1ac795913d88241171b45d796d3fe5dd38519d5a

I can install spdlog properly when running vcpkg locally, it's just Github Actions that's failing.

SamVanheer avatar Jun 21 '22 08:06 SamVanheer

Without this log, we can't get more information about Github Actions failure.😟

LilyWangLL avatar Jun 21 '22 09:06 LilyWangLL

As far as i can tell that's all that's actually in the log file. I tried to print the contents in three different ways (tried on Windows and Linux) and that's all that was printed.

The error appears to come from Git, the vcpkg code to issue the command is here: https://github.com/microsoft/vcpkg-tool/blob/47ed62a7d98b67433ef3d6315a90142918df5ebf/src/vcpkg/vcpkgpaths.cpp#L1096-L1107

But Git isn't providing enough information about why it's failing.

I enabled vcpkg debug logging like so:

set(VCPKG_VERBOSE ON)
set(VCPKG_INSTALL_OPTIONS "--debug" CACHE STRING "")

Before the CMake project call.

Here are the logs with the debug info added:

Windows log 
Run echo "[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] Failed to open: C:\Users\runneradmin\AppData\Local\vcpkg\config
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'manifests' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Failed to open: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\vcpkg-bundle.json
[DEBUG] Using scripts-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\scripts
[DEBUG] Bundle config: readonly=0, usegitregistry=0, embeddedsha=nullopt
[DEBUG] Using builtin-ports: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\ports
[DEBUG] Using installed-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\build\vcpkg_installed
[DEBUG] Using buildtrees-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees
[DEBUG] Using packages-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\packages
[DEBUG] Using manifest-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug
[DEBUG] Using vcpkg-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg
[DEBUG] Using scripts-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\scripts
[DEBUG] Using builtin-registry: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\versions
[DEBUG] Using downloads-root: D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\downloads
[DEBUG] Default binary cache path is: C:\Users\runneradmin\AppData\Local\vcpkg\archives
[DEBUG] Found path: C:\Program Files\CMake\bin\cmake.exe
[DEBUG] CreateProcessW(
C:\Program
Files\CMake\bin\cmake.exe --version)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 0 after    70709 us
[DEBUG] CreateProcessW(C:\Program
Files\CMake\bin\cmake.exe -DVCPKG_ROOT_DIR=D:/a/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg -DPACKAGES_DIR=D:/a/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/packages -DBUILDTREES_DIR=D:/a/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees -D_VCPKG_INSTALLED_DIR=D:/a/vcpkg-checkout-bug/vcpkg-checkout-bug/build/vcpkg_installed -DDOWNLOADS=D:/a/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/downloads -DVCPKG_MANIFEST_INSTALL=OFF -P D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees\0.vcpkg_dep_info.cmake)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 0 after    25322 us
[DEBUG] Found path: C:\Program Files\Git\bin\git.exe
[DEBUG] Found path: C:\Program Files\Git\cmd\git.exe
[DEBUG] Found path: C:\Program Files\Git\mingw64\bin\git.exe
[DEBUG] CreateProcessW(C:\Program
Files\Git\bin\git.exe --version)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 0 after    28274 us
[DEBUG] CreateProcessW(C:\Program
Files\Git\bin\git.exe --git-dir=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git --work-tree=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git -c core.autocrlf=false show 01ff40a106fe2f0549b89f89fb4c7c821bfb3c13:versions/baseline.json)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 0 after    36046 us
[DEBUG] CreateProcessW(C:\Program
Files\Git\bin\git.exe --git-dir=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git --work-tree=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git -c core.autocrlf=false archive 1ac795913d88241171b45d796d3fe5dd38519d5a -o D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees\versioning_\versions\spdlog\1ac795913d88241171b45d796d3fe5dd38519d5a.tar)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 128 after    39748 us
[DEBUG] CreateProcessW(C:\Program
Files\Git\bin\git.exe --git-dir=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git --work-tree=D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\.git -c core.autocrlf=false archive 3256ea88cc375fda2f977a2eb18435e23d498572 -o D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees\versioning_\versions\spdlog\3256ea88cc375fda2f977a2eb18435e23d498572.tar)
[DEBUG] ReadFile() finished with GetLastError(): 109
[DEBUG] 4404: cmd_execute_and_stream_data() returned 0 after    31063 us
[DEBUG] CreateProcessW(C:\Program
Files\CMake\bin\cmake.exe -E tar xzf D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees\versioning_\versions\spdlog\3256ea88cc375fda2f977a2eb18435e23d498572.tar)
[DEBUG] cmd_execute() returned 0 after 229967 us
Error: while checking out port spdlog with git tree 1ac795913d88241171b45d796d3fe5dd38519d5a
Error: Failed to tar port directory
error: tar failed with exit code: (128).
fatal: not a tree object: 1ac795913d88241171b45d796d3fe5dd38519d5a
[DEBUG] D:\a\_work\1\s\src\vcpkg\install.cpp(1122): 
[DEBUG] Time in subprocesses: 461129 us
[DEBUG] Time in parsing JSON: 40349 us
[DEBUG] Time in JSON reader: 1933 us
[DEBUG] Time in filesystem: 5066 us
[DEBUG] Time in loading ports: 341 us
[DEBUG] Exiting after 651.1 ms (567192 us)
Linux log 
[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] Failed to open: /home/runner/.vcpkg/config
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'manifests' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Failed to open: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/vcpkg-bundle.json
[DEBUG] Using scripts-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/scripts
[DEBUG] Bundle config: readonly=0, usegitregistry=0, embeddedsha=nullopt
[DEBUG] Using builtin-ports: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/ports
[DEBUG] Using installed-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/build/vcpkg_installed
[DEBUG] Using buildtrees-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees
[DEBUG] Using packages-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/packages
[DEBUG] Using manifest-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug
[DEBUG] Using vcpkg-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg
[DEBUG] Using scripts-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/scripts
[DEBUG] Using builtin-registry: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/versions
[DEBUG] Using downloads-root: /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/downloads
[DEBUG] Default binary cache path is: /home/runner/.cache/vcpkg/archives
[DEBUG] Found path: /usr/local/bin/cmake
[DEBUG] 1946: popen(/usr/local/bin/cmake --version 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 0 after     3463 us
[DEBUG] 1946: popen(/usr/local/bin/cmake -DVCPKG_ROOT_DIR=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg -DPACKAGES_DIR=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/packages -DBUILDTREES_DIR=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees -D_VCPKG_INSTALLED_DIR=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/build/vcpkg_installed -DDOWNLOADS=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/downloads -DVCPKG_MANIFEST_INSTALL=OFF -P /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees/0.vcpkg_dep_info.cmake 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 0 after     2844 us
[DEBUG] Found path: /usr/bin/git
[DEBUG] Found path: /bin/git
[DEBUG] 1946: popen(/usr/bin/git --version 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 0 after     1813 us
[DEBUG] 1946: popen(/usr/bin/git --git-dir=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git --work-tree=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git -c core.autocrlf=false show 01ff40a106fe2f0549b89f89fb4c7c821bfb3c13:versions/baseline.json 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 0 after     4642 us
[DEBUG] 1946: popen(/usr/bin/git --git-dir=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git --work-tree=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git -c core.autocrlf=false archive 1ac795913d88241171b45d796d3fe5dd38519d5a -o /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees/versioning_/versions/spdlog/1ac795913d88241171b45d796d3fe5dd38519d5a.tar 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 128 after     2610 us
[DEBUG] 1946: popen(/usr/bin/git --git-dir=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git --work-tree=/home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/.git -c core.autocrlf=false archive 3256ea88cc375fda2f977a2eb18435e23d498572 -o /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees/versioning_/versions/spdlog/3256ea88cc375fda2f977a2eb18435e23d498572.tar 2>&1)
[DEBUG] 1946: cmd_execute_and_stream_data() returned 0 after     2810 us
[DEBUG] system(cd /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees/versioning_/versions/spdlog/3256ea88cc375fda2f977a2eb18435e23d498572.tmp && /usr/local/bin/cmake -E tar xzf /home/runner/work/vcpkg-checkout-bug/vcpkg-checkout-bug/vcpkg/buildtrees/versioning_/versions/spdlog/3256ea88cc375fda2f977a2eb18435e23d498572.tar)
[DEBUG] cmd_execute() returned 0 after 84968 us
Error: while checking out port spdlog with git tree 1ac795913d88241171b45d796d3fe5dd38519d5a
Error: Failed to tar port directory
error: tar failed with exit code: (128).
fatal: not a tree object: 1ac795913d88241171b45d796d3fe5dd38519d5a
[DEBUG] /mnt/vss/_work/1/s/src/vcpkg/install.cpp(1122): 
[DEBUG] Time in subprocesses: 104005 us
[DEBUG] Time in parsing JSON: 21436 us
[DEBUG] Time in JSON reader: 979 us
[DEBUG] Time in filesystem: 932 us
[DEBUG] Time in loading ports: 88 us
[DEBUG] Exiting after 129.5 ms (128556 us)

Is this the information you need? Or do i need to pass any more command line arguments to enable more debugging options?

SamVanheer avatar Jun 21 '22 09:06 SamVanheer

I think i found the problem: both spdlog and Angelscript are listed in the dependencies array without a port version:

{
	"$schema": "https://raw.githubusercontent.com/microsoft/vcpkg/master/scripts/vcpkg.schema.json",
	"name": "halflife-unified-sdk",
	"version-semver": "0.1.0",
	"builtin-baseline": "b49b2bdf22559c89eddbf29a29621a1dc6929320",
	"dependencies": [
		{
			"name": "spdlog",
			"version>=": "1.10.0",
			"default-features": false
		},
		{
			"name": "nlohmann-json",
			"version>=": "3.10.5#3",
			"default-features": false
		},
		{
			"name": "angelscript",
			"version>=": "2.35.1",
			"default-features": false,
			"features": ["addons"]
		},
		{
			"name": "openal-soft",
			"version>=": "1.21.1#5",
			"default-features": false
		}
  ]
}

The documentation says that the port version is optional, but if i omit it i get the git checkout error.

I assume this means that if the port version is non-zero then it is required to specify it, but omitting it should cause an error to come from vcpkg itself and not Git, correct?

When i add a port version to spdlog in the test repository it works properly.

I initially omitted the port version because both projects are on port version 1 which i mistakenly assumed to mean the first version available in vcpkg (as in every update to the port in vcpkg increments it by one, which is incorrect). There was no error when running vcpkg locally so i didn't know this was incorrect.

It might be a good idea to put a link to the port-version documentation in the manifest documentation for the version field. Clarifying when the port version is needed will help prevent this mistake from being made as well.

SamVanheer avatar Jun 21 '22 19:06 SamVanheer

Thanks for your reply. According to the following log, if you don't set port-version in vcpkg.json, it will download both port-version=0 and port-version=1 of spdlog, and it failed to checkout port-version=0 of spdlog.

[DEBUG] CreateProcessW(C:\Program
Files\CMake\bin\cmake.exe -E tar xzf D:\a\vcpkg-checkout-bug\vcpkg-checkout-bug\vcpkg\buildtrees\versioning_\versions\spdlog\3256ea88cc375fda2f977a2eb18435e23d498572.tar)
[DEBUG] cmd_execute() returned 0 after 229967 us
Error: while checking out port spdlog with git tree 1ac795913d88241171b45d796d3fe5dd38519d5a
Error: Failed to tar port directory
error: tar failed with exit code: (128).
fatal: not a tree object: 1ac795913d88241171b45d796d3fe5dd38519d5a

The versions of spdlog:

{
  "versions": [
    {
      "git-tree": "3256ea88cc375fda2f977a2eb18435e23d498572",
      "version-semver": "1.10.0",
      "port-version": 1
    },
    {
      "git-tree": "1ac795913d88241171b45d796d3fe5dd38519d5a",
      "version-semver": "1.10.0",
      "port-version": 0
    },

@ras0219-msft Could you help take a look this issue? This issue looks like a vcpkg bug.

LilyWangLL avatar Jun 22 '22 09:06 LilyWangLL

I just found the cause of the problem. In Github Actions, by default, actions/checkout only checks out the last commit. Port version 0 doesn't exist in the latest version of these ports so vcpkg tries to get an older commit. Since the older commits aren't checked out it fails to get that commit.

Modifying the CI configuration to fetch all commits fixes this:

name: CI/CD

on: [push, pull_request]

jobs:
  linux:
    name: Linux-x86
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          submodules: recursive
          fetch-depth: 0
      - name: Configure
        run: cmake -B ${{github.workspace}}/build -DCMAKE_TOOLCHAIN_FILE=${{github.workspace}}/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET="x64-linux"
        continue-on-error: true
      - name: Print vcpkg log
        run: cat "${{github.workspace}}/build/vcpkg-manifest-install.log"
      - name: Build
        run: cmake --build ${{github.workspace}}/build
  win32:
    name: Win32
    runs-on: windows-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          submodules: recursive
          fetch-depth: 0
      - name: Configure
        run: cmake -B ${{github.workspace}}/build -DCMAKE_TOOLCHAIN_FILE=${{github.workspace}}/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET="x86-windows" -A Win32
        continue-on-error: true
      - name: Print vcpkg log
        run: type "${{github.workspace}}/build/vcpkg-manifest-install.log"
      - name: Build
        run: cmake --build ${{github.workspace}}/build

The problem is that this adds a lot of overhead. Checking out all commits is expensive and slow, getting slower for each commit (and this includes vcpkg commits).

Perhaps vcpkg should check if the commit that a port version is associated with is checked out, and report failure if it can't be found?

Here's the Github Actions run for fetch-depth: 0 in case you need it: https://github.com/SamVanheer/vcpkg-checkout-bug/actions/runs/2541219642

Note that i added nlohmann_json as a dependency to test the difference.

SamVanheer avatar Jun 22 '22 09:06 SamVanheer

Does this require more information or steps to reproduce locally?

SamVanheer avatar Jun 26 '22 09:06 SamVanheer

Does this require more information or steps to reproduce locally?

Nothing, thanks for your work, we will dicussion this issue on our meeting in this week.

LilyWangLL avatar Jun 27 '22 01:06 LilyWangLL

Port versions don't use commit hashes, we use the git tree hash of the port directory. As far as I know, there is no way to deduce the commit that added a specific tree hash.

Unfortunately, this means there is nothing we can do in the product itself to solve this problem.

vicroms avatar Jun 27 '22 20:06 vicroms

Perhaps checking if the vcpkg repository is a shallow repository could help? The checkout action appears to create a shallow repository if fetch-depth is > 0 so this should show if the vcpkg repository only had its last commit checked out.

Alternatively this could be used to check if only the last commit was checked out.

If Git fails to return the requested information then the result of these commands could be used to alert the user to the cause.

It wouldn't catch cases where a few commits but not all were checked out, but since this is most likely to occur in a CI build where only one commit is checked out it would cover the most common case.

SamVanheer avatar Jun 27 '22 21:06 SamVanheer

Yeah, I agree that improving our diagnostics is something we should do. Truly thanks for looking into this issue!

Do you feel like creating a PR to add this validation? If not, I'll look into this during the week.

vicroms avatar Jun 27 '22 22:06 vicroms

I'll leave it up to you. I haven't worked with this codebase before.

SamVanheer avatar Jun 29 '22 21:06 SamVanheer

I've experienced the same problem in Gitlab with Gitlab Runner on a Windows VM.

Due to a bug in Gitlab, the flag GIT_DEPTH=0 did not prevent shallow copies. Adding GIT_DEPTH=0 together with GIT_STRATEGY=clone functions as a workaround for me.

jnslmk avatar Aug 09 '22 08:08 jnslmk

I see this problem with Boost too. I used overrides in my manifest to drop a load of Boost modules to version to 1.74, works fine locally (because of the complete git history) but fails in a Github Action due to a fetch-depth of 1 (thanks to @SamVanheer for finding that). Changing my yml to get the complete history fixes the problem, but it's inefficient.

cmannett85 avatar Sep 03 '22 10:09 cmannett85

I am experiencing the same problem with Azure Devops and still haven't found a method to fix this problem.

jbbjarnason avatar Oct 21 '22 14:10 jbbjarnason

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

github-actions[bot] avatar Dec 21 '23 01:12 github-actions[bot]