Cannot use vcpkg in manifest mode with cpp image

[NoMore201]

When using vcpkg in manifest mode, it is possible to specify a commit of the vcpkg repository to track via the builtin-baseline option, such as:

{
  "name": "test-vcpkg-template",
  "dependencies": [
    {
      "name": "expected-lite",
      "version>=": "0.6.3"
    }
  ],
  "builtin-baseline": "01acfdfde3ed99280d3883a8fccd5fa4408f5214"
}

Devcontainer cpp image clones vcpkg repository using the --depth=1 option (link to the code) preventing usage of vcpkg in manifest mode, unless manually modifying the container image to fetch all git history.

[samruddhikhandale]

Hi 👋

Thanks for opening the issue. To keep the cpp image size minimal, we are intentionally using --depth=1. However, I understand how the builtin-baseline option might become unusable due to this.

Doing a full git clone would significantly increase the size of the cpp image. @NoMore201, do you think cloning up to the last N commits would be useful to you?

[NoMore201]

Hello!

To be fair I'm not a vcpkg expert to tell if this is the right way to do things, since no one else complained so far. Anyway it is rather easy to modify the dev container to clone the entire history.

I would say to keep it like it is, unless someone else with more experience has something to add.

[Thomas1664]

@samruddhikhandale
For vcpkg to work correctly, it must not be shallow-cloned.
If you ant to keep the container size small, I suggest adding vcpkg as a feature. This keeps image size down and enables vcpkg users to use vcpkg properly at the expense of a larger container.

I recommend you get in touch with the vcpkg team at Microsoft if you have further questions.

[willie]

I just spent several hours debugging a working config that I was moving to dev containers because the installed vcpkg is broken for manifest mode. It makes no sense to intentionally hobble a specific mode and not document it in the readme.

[Kaniska244]

Hello @willie ,

Thank you for raising your concern on this issue. Would you kindly let me know at which stage(while installing the packages using vcpkg in manifest mode or while building the project later) does this fail for you & also if possible would you kindly share the respective manifest file used for this instance?
That could help me a great deal to bring this issue to a conclusion. There's a 3rd party feature available in devcontainer feature site but even there they are getting the vcpkg from git in shallow copy mode.

With Regards,
Kaniska

[willie]

In manifest mode, you add vcpkg to the CMakeLists.txt:

set(CMAKE_TOOLCHAIN_FILE "/usr/local/vcpkg/scripts/buildsystems/vcpkg.cmake" CACHE STRING "vcpkg toolchain file")
and then when you generate the build files, vcpkg reads the manifest and builds then. It's like the manual install step, but it's triggered by the manifest in this step

cmake -B out -G Ninja -D CMAKE_BUILD_TYPE=Debug

At that point, the shallow copy gives you all kinds errors.

I had to add this to my devcontainer dockerfile (VCPKG_FORCE_SYSTEM_BINARIES is for arm64):

# vcpkg
ENV VCPKG_FORCE_SYSTEM_BINARIES=true
RUN apt-get -y update && export DEBIAN_FRONTEND=noninteractive && apt-get install -y \
    autoconf automake autoconf-archive
RUN cd /usr/local/vcpkg && git fetch --unshallow && git pull --ff-only && ./bootstrap-vcpkg.sh

The docs for vcpkg install are very clear on how to install it.

[Kaniska244]

Hello @willie ,

Thank you so much for the detailed explanation. I tried it with the following steps with the cpp devcontainer image. For this at least I didn't need to clone the complete vcpkg repository although I did it in manifest mode, It was only needed to install some extra system libraries as given below which I added in the Dockerfile. Perhaps there are more complicated cases on which I am missing out here. Kindly let me know in case of such concerns.

1. Added these additional system libraries autoconf automake libtool m4 autoconf-archive in the Dockerfile & built the cpp image.
2. Started container from the image & Inside the cpp container created a sample project called test-project with manifest file vcpkg.json & also a small source file main.cpp along with CMakeLists.txt as below.

root ➜ / $ cd test-project
root ➜ /test-project $ uname -m
x86_64
root ➜ /test-project $ 
root ➜ /test-project $ 
root ➜ /test-project $ cat CMakeLists.txt
cmake_minimum_required(VERSION 3.0.0)
project(test-project VERSION 1.0.0)

include(CTest)
enable_testing()

add_executable(test-project main.cpp)

set(CPACK_PROJECT_NAME ${PROJECT_NAME})
set(CPACK_PROJECT_VERSION ${PROJECT_VERSION})
set(CMAKE_TOOLCHAIN_FILE "/usr/local/vcpkg/scripts/buildsystems/vcpkg.cmake")
include(CPack)

root ➜ /test-project $ 
root ➜ /test-project $ cat main.cpp
/*-------------------------------------------------------------------------------------------------------------
 * Copyright (c) Microsoft Corporation. All rights reserved.
 * Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
 *-------------------------------------------------------------------------------------------------------------*/

#include <iostream>
using namespace std;

int main() 
{
    cout << "Hello, Remote World!" << "\n";
    return 0;
}root ➜ /test-project $ cat vcpkg.json
{
  "name": "test-project",
  "version-string": "1.0.0",
  "dependencies": [
    "boost",
    "fmt"
  ]
}

root ➜ /test-project $ pwd
/test-project

3. Started the package installation in manifest mode with vcpkg install as below.

root ➜ /test-project $ vcpkg install > install.out 

root ➜ /test-project $ 
root ➜ /test-project $ tail -50f install.out
Successfully downloaded boostorg-accumulators-boost-1.87.0.tar.gz
-- Extracting source /usr/local/vcpkg-downloads/boostorg-accumulators-boost-1.87.0.tar.gz
-- Using source at /usr/local/vcpkg/buildtrees/boost-accumulators/src/ost-1.87.0-49ca054607.clean
-- Configuring x64-linux
-- Building x64-linux-rel
-- Installing: /usr/local/vcpkg/packages/boost-accumulators_x64-linux/share/boost-accumulators/copyright
-- Performing post-build validation
Starting submission of boost-accumulators:[email protected] to 1 binary cache(s) in the background
Elapsed time to handle boost-accumulators:x64-linux: 1.7 s
boost-accumulators:x64-linux package ABI: 1dd6f6e5c6a6c7c0844586de2e4e152b0cce56477ae01e5e114d2d94b7876d1d
Completed submission of boost-assign:[email protected] to 1 binary cache(s) in 4.52 ms
Installing 169/170 boost:[email protected]#1...
Building boost:[email protected]#1...
-- Skipping post-build validation due to VCPKG_POLICY_EMPTY_PACKAGE
Starting submission of boost:[email protected]#1 to 1 binary cache(s) in the background
Elapsed time to handle boost:x64-linux: 28.7 ms
boost:x64-linux package ABI: 7d5b7a293781d295149ff77e8ba937a470f2888718382217d090f3b6170e16f5
Completed submission of boost-accumulators:[email protected] to 1 binary cache(s) in 16.1 ms
Installing 170/170 fmt:[email protected]#1...
Building fmt:[email protected]#1...
Downloading https://github.com/fmtlib/fmt/archive/11.0.2.tar.gz -> fmtlib-fmt-11.0.2.tar.gz
Successfully downloaded fmtlib-fmt-11.0.2.tar.gz
-- Extracting source /usr/local/vcpkg-downloads/fmtlib-fmt-11.0.2.tar.gz
-- Applying patch fix-write-batch.patch
-- Applying patch fix-pass-utf-8-only-if-the-compiler-is-MSVC-at-build.patch
-- Using source at /usr/local/vcpkg/buildtrees/fmt/src/11.0.2-c30c0a133f.clean
-- Configuring x64-linux
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Fixing pkgconfig file: /usr/local/vcpkg/packages/fmt_x64-linux/lib/pkgconfig/fmt.pc
-- Fixing pkgconfig file: /usr/local/vcpkg/packages/fmt_x64-linux/debug/lib/pkgconfig/fmt.pc
-- Installing: /usr/local/vcpkg/packages/fmt_x64-linux/share/fmt/usage
-- Installing: /usr/local/vcpkg/packages/fmt_x64-linux/share/fmt/copyright
-- Performing post-build validation
Starting submission of fmt:[email protected]#1 to 1 binary cache(s) in the background
Elapsed time to handle fmt:x64-linux: 6.9 s
fmt:x64-linux package ABI: 6d1460c254f4f064377556e624bd01f23e07890e429e638e2fb42b44f3463ab6
Total install time: 12 min
The package fmt provides CMake targets:

    find_package(fmt CONFIG REQUIRED)
    target_link_libraries(main PRIVATE fmt::fmt)

    # Or use the header-only version
    find_package(fmt CONFIG REQUIRED)
    target_link_libraries(main PRIVATE fmt::fmt-header-only)

Completed submission of boost:[email protected]#1 to 1 binary cache(s) in 2.56 ms
Waiting for 1 remaining binary cache submissions...
Completed submission of fmt:[email protected]#1 to 1 binary cache(s) in 93.9 ms (1/1)
^Z
[1]+  Stopped                 tail -50f install.out
root ➜ /test-project $

5. Next to integrate vcpkg with CMake project executed the below.

root ➜ /test-project $ cd /usr/local/vcpkg
root ➜ /usr/local/vcpkg (master) $ ./vcpkg integrate install
Applied user-wide integration for this vcpkg root.
CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=/usr/local/vcpkg/scripts/buildsystems/vcpkg.cmake"
root ➜ /usr/local/vcpkg (master) $ cd /test-project
root ➜ /test-project $ 

7. Followed by that built the test-project as below.

root ➜ /test-project $ mkdir build
root ➜ /test-project $ cd build
root ➜ /test-project/build $ ls -lha
total 8.0K
drwxr-xr-x 2 root root 4.0K Mar 20 08:50 .
drwxr-xr-x 4 root root 4.0K Mar 20 08:50 ..
root ➜ /test-project/build $ cmake ..
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /test-project/build
root ➜ /test-project/build $ 
root ➜ /test-project/build $ ls -lha
total 76K
drwxr-xr-x  4 root root 4.0K Mar 20 08:51 .
drwxr-xr-x  4 root root 4.0K Mar 20 08:50 ..
-rw-r--r--  1 root root  19K Mar 20 08:51 CMakeCache.txt
drwxr-xr-x 34 root root 4.0K Mar 20 08:51 CMakeFiles
-rw-r--r--  1 root root 3.5K Mar 20 08:51 CPackConfig.cmake
-rw-r--r--  1 root root 3.9K Mar 20 08:51 CPackSourceConfig.cmake
-rw-r--r--  1 root root  245 Mar 20 08:51 CTestTestfile.cmake
-rw-r--r--  1 root root 2.4K Mar 20 08:51 DartConfiguration.tcl
-rw-r--r--  1 root root  20K Mar 20 08:51 Makefile
drwxr-xr-x  3 root root 4.0K Mar 20 08:51 Testing
-rw-r--r--  1 root root 1.6K Mar 20 08:51 cmake_install.cmake
root ➜ /test-project/build $ cmake --build .
[ 50%] Building CXX object CMakeFiles/test-project.dir/main.cpp.o
[100%] Linking CXX executable test-project
[100%] Built target test-project
root ➜ /test-project/build $ 

9. Finally executed the code from the executable created.

root ➜ /test-project/build $ ls -lha
total 92K
drwxr-xr-x  4 root root 4.0K Mar 20 08:51 .
drwxr-xr-x  4 root root 4.0K Mar 20 08:50 ..
-rw-r--r--  1 root root  19K Mar 20 08:51 CMakeCache.txt
drwxr-xr-x 34 root root 4.0K Mar 20 08:51 CMakeFiles
-rw-r--r--  1 root root 3.5K Mar 20 08:51 CPackConfig.cmake
-rw-r--r--  1 root root 3.9K Mar 20 08:51 CPackSourceConfig.cmake
-rw-r--r--  1 root root  245 Mar 20 08:51 CTestTestfile.cmake
-rw-r--r--  1 root root 2.4K Mar 20 08:51 DartConfiguration.tcl
-rw-r--r--  1 root root  20K Mar 20 08:51 Makefile
drwxr-xr-x  3 root root 4.0K Mar 20 08:51 Testing
-rw-r--r--  1 root root 1.6K Mar 20 08:51 cmake_install.cmake
-rwxr-xr-x  1 root root  16K Mar 20 08:51 test-project
root ➜ /test-project/build $ 
root ➜ /test-project/build $ ./test-project
Hello, Remote World!
root ➜ /test-project/build $