[Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. #14883

ericoporto · 2020-12-01T21:36:59Z

Describe the bug
Error when building vcpkg current master f8baf48

Environment

OS: Windows Server Core 2019
Compiler: VISUALCPP_BUILD_TOOLS_VERSION=14.0.25420.1

To Reproduce
My Dockerfile:

# base will already have Chocolatey installed
FROM cirrusci/windowsservercore:2019

# no temp folder exists by default
RUN mkdir %TEMP%

ARG VISUALCPP_BUILD_TOOLS_VERSION=14.0.25420.1
RUN cinst visualcpp-build-tools --version %VISUALCPP_BUILD_TOOLS_VERSION% -y

# Windows 8.0 SDK (but only install the .NET 4.5 SDK)
RUN cd %TEMP% && \
  curl -fLO http://download.microsoft.com/download/F/1/3/F1300C9C-A120-4341-90DF-8A52509B23AC/standalonesdk/sdksetup.exe && \
  start /wait sdksetup /ceip off /features OptionID.NetFxSoftwareDevelopmentKit /quiet /norestart

ARG NUGET_VERSION=5.7.0
RUN cinst nuget.commandline --version %NUGET_VERSION% -y

ARG INNO_SETUP_VERSION=6.0.5
RUN cinst innosetup --version %INNO_SETUP_VERSION% -y

ARG CMAKE_VERSION=3.18.4
RUN cinst cmake --version %CMAKE_VERSION% --installargs ADD_CMAKE_TO_PATH=System -y

RUN pushd %TEMP% && \
  git clone https://github.com/Microsoft/vcpkg.git && \
  pushd vcpkg && \
  if not exist vcpkg.exe call bootstrap-vcpkg.bat && \
  vcpkg install libogg:x86-windows-static libtheora:x86-windows-static libvorbis:x86-windows-static && \
  popd && \
  popd && \
  mkdir Lib\Xiph && \
  copy %TEMP%\vcpkg\installed\x86-windows-static\lib\ogg.lib Lib\Xiph\libogg_static.lib && \
  copy %TEMP%\vcpkg\installed\x86-windows-static\lib\theora.lib Lib\Xiph\libtheora_static.lib && \
  copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbis.lib Lib\Xiph\libvorbis_static.lib && \
  copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbisfile.lib Lib\Xiph\libvorbisfile_static.lib

This gives me the following error

C:\Users\ContainerAdministrator\AppData\Local\Temp\vcpkg\scripts\bootstrap.ps1 
: Building vcpkg.exe failed. Please ensure you have installed Visual Studio 
with the Desktop C++ workload and the Windows SDK for Desktop C++.
At line:1 char:4
+ & {& 'C:\Users\ContainerAdministrator\AppData\Local\Temp\vcpkg\script ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorExcep 
   tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio 
   n,bootstrap.ps1
 
ScriptHalted
At C:\Users\ContainerAdministrator\AppData\Local\Temp\vcpkg\scripts\bootstrap.p
s1:411 char:5
+     throw
+     ~~~~~
    + CategoryInfo          : OperationStopped: (:) [], RuntimeException
    + FullyQualifiedErrorId : ScriptHalted
 
The command 'cmd /S /C pushd %TEMP% &&   git clone https://github.com/Microsoft/vcpkg.git &&   pushd vcpkg &&   if not exist vcpkg.exe call bootstrap-vcpkg.bat &&   vcpkg install libogg:x86-windows-static libtheora:x86-windows-static libvorbis:x86-windows-static &&   popd &&   popd &&   mkdir Lib\Xiph &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\ogg.lib Lib\Xiph\libogg_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\theora.lib Lib\Xiph\libtheora_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbis.lib Lib\Xiph\libvorbis_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbisfile.lib Lib\Xiph\libvorbisfile_static.lib' returned a non-zero code: 1
Repro code when

Expected behavior
I expected vcpkg to build fine, it was building fine at least yesterday.

Additional context
This build is running in Cirrus CI

The text was updated successfully, but these errors were encountered:

JackBoosY · 2020-12-02T02:58:38Z

The command 'cmd /S /C pushd %TEMP% &&   git clone https://github.com/Microsoft/vcpkg.git &&   pushd vcpkg &&   if not exist vcpkg.exe call bootstrap-vcpkg.bat &&   vcpkg install libogg:x86-windows-static libtheora:x86-windows-static libvorbis:x86-windows-static &&   popd &&   popd &&   mkdir Lib\Xiph &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\ogg.lib Lib\Xiph\libogg_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\theora.lib Lib\Xiph\libtheora_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbis.lib Lib\Xiph\libvorbis_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbisfile.lib Lib\Xiph\libvorbisfile_static.lib' returned a non-zero code: 1

Can you run the following command manually and provide the failure log?

cmd /S /C pushd %TEMP% &&   git clone https://github.com/Microsoft/vcpkg.git &&   pushd vcpkg &&   if not exist vcpkg.exe call bootstrap-vcpkg.bat &&   vcpkg install libogg:x86-windows-static libtheora:x86-windows-static libvorbis:x86-windows-static &&   popd &&   popd &&   mkdir Lib\Xiph &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\ogg.lib Lib\Xiph\libogg_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\theora.lib Lib\Xiph\libtheora_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbis.lib Lib\Xiph\libvorbis_static.lib &&   copy %TEMP%\vcpkg\installed\x86-windows-static\lib\vorbisfile.lib Lib\Xiph\libvorbisfile_static.lib

Thanks.

ericoporto · 2020-12-02T10:25:34Z

@JackBoosY Hey, it builds fine on my personal Windows 10 Home machine -> gist:vcpkg_2020_12_02.log

But I have full Visual Studio 2019 with everything installed. I need to be able to build vcpkg on a CI system, and the mentioned minimal docker image above used to build fine until very recently. Unfortunately, I can't seem to be able to run Docker (Windows based) on my personal machine, so my Docker interaction has been by trying things directly with the CI system.

The command that I need to work with can be just

cmd /S /C pushd %TEMP% &&   git clone https://github.com/Microsoft/vcpkg.git &&   pushd vcpkg &&   if not exist vcpkg.exe call bootstrap-vcpkg.bat

Maybe the minimal requirements for the visualcpp-build-tools changed but I don't know what they are now.

ericoporto · 2020-12-02T13:02:20Z

I think I found the Dockerfile I build upon, it uses mcr.microsoft.com/windows/servercore:ltsc2019 as the base.

I understand the Readme says : Visual Studio 2015 Update 3 or greater with the English language pack; but I really wanted to use vcpkg in a CI environment on Windows Docker, is there anything I can add to fill current visual studio requirements for vcpkg?

My cpp project needs to keep compatibility with Windows 7 and forward.

I tried to add

RUN cinst -y --no-progress visualstudio2019community

On the dockerfile, but this requires the machine to restart, breaking the Dockerfile build process.

I tried to instead add

RUN cinst -y --no-progress visualstudio2019-workload-nativedesktop

But this didn't work either, because it can't find the full VS installation.

Is there a package for vs2019 that is safe to use in a Dockerfile build?

I found this online but it's old and doesn't work - and was closed as not priority!

JackBoosY · 2020-12-03T02:14:15Z

cc @BillyONeal

ericoporto · 2020-12-03T10:08:08Z

Sorry, turns out it's a change of components on the base Docker image on my CI provider. This is the minimum I needed to build vcpkg:

FROM cirrusci/windowsservercore:2019

# Microsoft .NET Framework 4.8 4.8.0.20190930 is broken and causes the
# installation of visualstudio2019-workload-vctools to fail. To workaround
# the issue request Microsoft .NET Framework 4.7.2.20180712 instead, which
# is already installed and thus speeds up installation as well.
# this is just to prevent visualstudio2019 to force a reset in the machine
RUN cinst -y --no-progress dotnetfx --version=4.7.2.20180712

RUN cinst -y --no-progress --version=16.8.2.0 visualstudio2019buildtools    
RUN cinst -y --no-progress --version=1.0.0 visualstudio2019-workload-vctools    
RUN cinst -y --no-progress --version=1.0.0 visualstudio2019-workload-nativedesktop || echo "ignore reboot request"

ARG CMAKE_VERSION=3.14.5
RUN cinst cmake --version %CMAKE_VERSION% --installargs ADD_CMAKE_TO_PATH=System -y

It takes an hour to build this Docker but after it can be used to build vcpkg. This can be closed now.

ericoporto · 2020-12-03T10:09:00Z

More details are here: cirruslabs/docker-images-windows#16

Sorry for the confusion.

BillyONeal · 2020-12-03T18:54:17Z

It looks like the actual cause was that installing the build tools failed and thus there was no bug? Can you confirm @ericoporto ?

ericoporto · 2020-12-03T18:57:31Z

Yes, it was what I concluded in the end. I still left the Dockerfile in the comments in case someone find themselves working with cpp in the mcr.microsoft.com/windows/servercore:ltsc2019 sincr there isn't much written about this online and all on VS online assumes the IDE or an environment with a GUI.

Note: there wasn't any error on any command line anywhere before but the vcpkg.

ericoporto · 2020-12-11T01:53:27Z

Sorry, after a long investigation I do believe something changed with vcpkg bootstrapping.
What are the minimum VS components / workload to bootstrapping vcpkg ? From here: https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019#visual-c-build-tools

I am pretty sure I didn't needed everything from Microsoft.VisualStudio.Workload.VCTools workload not too long ago.

Is there any chance it used to work with standalone CMake and now requires Microsoft.VisualStudio.Component.VC.CMake.Project ?

BillyONeal · 2020-12-11T16:51:04Z

I think it's more likely that how VS factored their components changed, since we haven't changed anything about bootstrap but our CI has been broken by VS changing their component structures before.

Is there any chance it used to work with standalone CMake and now requires Microsoft.VisualStudio.Component.VC.CMake.Project ?

Bootstrap does not use cmake at all, it uses https://github.com/microsoft/vcpkg/blob/master/toolsrc/windows-bootstrap/vcpkg.vcxproj

ericoporto · 2020-12-11T17:51:12Z

Thanks! I am following the guide here: https://docs.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2019

And trying to figure out the exact minimal things to be able to figure out why it broke

I checked the details of the previous image and I was using a container with some workloads installed (including recommended but not optional), but the image was fairly old - around an year maybe, maybe even a bit more? Recently the image dropped from the cache and the new Docker container even though theoretically didn't change VS install wise (other things changed like mingw and msys2) , vcpkg stopped building.

The reason I am investigating is because relying only on Workloads my container that successfully bootstraps vcpkg right now is a bit too big, leading to timeouts on my CI.

Thanks for directing me to the proper vcxproj since this will make things a lot easier to me!

PhoebeHui · 2021-03-05T01:07:48Z

Building vcpkg.exe is not a problem now, with latest source, it will downlaod the latest release version of vcpkg on windows, we don't need build it again. Please reopen this issue if I misunderstand someting here.

DemetriouJohn · 2021-04-06T14:16:03Z

I am having similar issues with running vcpkg on the latest windows server core 20H2 docker image.
It tries to find MSBuild but for some reasons searches in the wrong paths

PhoebeHui · 2021-04-07T00:38:43Z

@DevsAnon, please creat a new issue to track the failures.

ericoporto changed the title ~~Building vcpkg.exe failed. in Windows Server Core 2019~~ Building vcpkg.exe failed. in Windows Server Core 2019. It built fine yesterday. Attached Dockerfile. Dec 1, 2020

ericoporto changed the title ~~Building vcpkg.exe failed. in Windows Server Core 2019. It built fine yesterday. Attached Dockerfile.~~ Building vcpkg.exe failed suddenly in Windows Server Core 2019. Dec 1, 2020

JonLiu1993 assigned JackBoosY Dec 2, 2020

JackBoosY added the category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) label Dec 2, 2020

ericoporto mentioned this issue Dec 2, 2020

Building vcpkg.exe failed suddenly in Windows Server Core 2019 cirruslabs/docker-images-windows#16

Closed

JackBoosY changed the title ~~Building vcpkg.exe failed suddenly in Windows Server Core 2019.~~ [Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. Dec 3, 2020

JackBoosY assigned BillyONeal Dec 3, 2020

ericoporto closed this as completed Dec 3, 2020

JackBoosY added category:question This issue is a question and removed category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) labels Dec 4, 2020

ericoporto reopened this Dec 11, 2020

ericoporto mentioned this issue Dec 11, 2020

Feature Request: make vcpkg embeddable in docker image. #645

Closed

PhoebeHui closed this as completed Mar 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. #14883

[Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. #14883

ericoporto commented Dec 1, 2020

JackBoosY commented Dec 2, 2020

ericoporto commented Dec 2, 2020 •

edited

Loading

ericoporto commented Dec 2, 2020 •

edited

Loading

JackBoosY commented Dec 3, 2020

ericoporto commented Dec 3, 2020 •

edited

Loading

ericoporto commented Dec 3, 2020

BillyONeal commented Dec 3, 2020

ericoporto commented Dec 3, 2020 •

edited

Loading

ericoporto commented Dec 11, 2020 •

edited

Loading

BillyONeal commented Dec 11, 2020

ericoporto commented Dec 11, 2020

PhoebeHui commented Mar 5, 2021

DemetriouJohn commented Apr 6, 2021

PhoebeHui commented Apr 7, 2021

[Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. #14883

[Docker] Building vcpkg.exe failed suddenly in Windows Server Core 2019. #14883

Comments

ericoporto commented Dec 1, 2020

JackBoosY commented Dec 2, 2020

ericoporto commented Dec 2, 2020 • edited Loading

ericoporto commented Dec 2, 2020 • edited Loading

JackBoosY commented Dec 3, 2020

ericoporto commented Dec 3, 2020 • edited Loading

ericoporto commented Dec 3, 2020

BillyONeal commented Dec 3, 2020

ericoporto commented Dec 3, 2020 • edited Loading

ericoporto commented Dec 11, 2020 • edited Loading

BillyONeal commented Dec 11, 2020

ericoporto commented Dec 11, 2020

PhoebeHui commented Mar 5, 2021

DemetriouJohn commented Apr 6, 2021

PhoebeHui commented Apr 7, 2021

ericoporto commented Dec 2, 2020 •

edited

Loading

ericoporto commented Dec 2, 2020 •

edited

Loading

ericoporto commented Dec 3, 2020 •

edited

Loading

ericoporto commented Dec 3, 2020 •

edited

Loading

ericoporto commented Dec 11, 2020 •

edited

Loading