Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nettle:x64-windows] build failure #12240

Open
ghost opened this issue Jul 3, 2020 · 24 comments
Open

[nettle:x64-windows] build failure #12240

ghost opened this issue Jul 3, 2020 · 24 comments
Assignees
Labels
category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly

Comments

@ghost
Copy link

ghost commented Jul 3, 2020

Host Environment

  • OS: [Microsoft Windows [Version 10.0.19041.331]]
  • Compiler:
    • Visual Studio 2017 Version: 15.9.24 VS Tools Version: 14.16.27023
    • Visual Studio 2019 Version: 16.6.3 VS Tools Version: 14.26.28801

To Reproduce
Steps to reproduce the behavior:
vcpkg install nettle[core]:x64-windows

Failure logs

  • Cut and past the appropriate build messages from the console output.
    Computing installation plan...
    The following packages will be built and installed:
    nettle[core]:x64-windows
    Starting package 1/1: nettle:x64-windows
    Building package nettle[core]:x64-windows...
    -- Using cached D:/Git/vcpkg.downloads/ShiftMediaProject-nettle-1d0a6e64e01458fdf37eaf5d90975deb52c3da41.tar.gz
    -- Using source at D:/Git/vs2019.prod/buildtrees/nettle/src/eb52c3da41-e03f56976f
    -- Building SMP/libnettle.sln for Release
    CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message):
    Command failed: msbuild D:/Git/vs2019.prod/buildtrees/nettle/x64-windows-rel/eb52c3da41-e03f56976f/SMP/libnettle.sln /p:Configuration=ReleaseDLL /p:UseEnv=True /t:Rebuild /p:Platform=x64 /p:PlatformToolset=v142 /p:VCPkgLocalAppDataDisabled=true /p:UseIntelMKL=No /p:WindowsTargetPlatformVersion=10.0.19041.0 /p:VcpkgTriplet=x64-windows /p:VcpkgCurrentInstalledDir=D:/Git/vs2019.prod/installed/x64-windows /p:VcpkgManifestInstall=false /m /p:ForceImportBeforeCppTargets=D:/Git/vs2019.prod/scripts/buildsystems/msbuild/vcpkg.targets /p:VcpkgApplocalDeps=false
    Working Directory: D:/Git/vs2019.prod/buildtrees/nettle/x64-windows-rel/eb52c3da41-e03f56976f
    Error code: 1
    See logs for more information:
    D:\Git\vs2019.prod\buildtrees\nettle\build-x64-windows-rel-out.log

Call Stack (most recent call first):
scripts/cmake/vcpkg_install_msbuild.cmake:162 (vcpkg_execute_required_process)
ports/nettle/portfile.cmake:66 (vcpkg_install_msbuild)
scripts/ports.cmake:76 (include)

Error: Building package nettle:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: nettle:x64-windows
Vcpkg version: 2020.02.04-nohash

Additionally, attach any relevant sections from the log files above.

Additional context
This may/may not have relevance from issue [<gmp>] build failure referenced #12237 as I have a work around patch to place the gmp.h and gmpxx.h in the <vcpkg-port>\installed\x64-windows\include folder instead of the <vcpkg-port>\installed\x64-windows\include\include folder.

@JackBoosY JackBoosY changed the title [nettle>] build failure [nettle:x64-windows] build failure Jul 3, 2020
@JackBoosY JackBoosY self-assigned this Jul 3, 2020
@JackBoosY JackBoosY added the category:port-bug The issue is with a library, which is something the port should already support label Jul 3, 2020
@JackBoosY
Copy link
Contributor

I know this bug, @Neumann-A could you please take a look?

Thanks.

@Neumann-A
Copy link
Contributor

Probably the segfault in yasm we don't know why it occours. Updating VS somehow resolves the problem.

@Neumann-A
Copy link
Contributor

I am also unable to reproduce the error with Visual Studio 2019 Version: 16.6.3 locally

@JackBoosY
Copy link
Contributor

@Neumann-A Yes, we can't repro this issue with Visual Studio 2019, and this PR is the same as some PR I encountered.

@MVoz
Copy link
Contributor

MVoz commented Jul 3, 2020

open ...\x64-windows\share\vs-yasm\yasm.props

  <PropertyGroup>
    <YasmPath Condition= "'$(YASMPATH)' == ''"></YasmPath>
  </PropertyGroup>
  <ItemDefinitionGroup>
    <YASM>
      <Preprocessor>0</Preprocessor>
      <Parser>0</Parser>
      <ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>
      <CommandLineTemplate Condition="'$(Platform)' == 'Win32'">"$(YasmPath)"yasm.exe -Xvc -f win32 [AllOptions] [AdditionalOptions] "%(FullPath)"

"$(YasmPath)"yasm.exe <<--

''"></YasmPath> <<-- null

edit to
<YasmPath Condition= "'$(YASMPATH)' == ''">$(VCPKG_ROOT)/download/bla/bla/yasm_path</YasmPath>


or delete "$(YasmPath)"

"$(YasmPath)"yasm.exe -Xvc -f win32 [AllOptions] -->> yasm.exe -Xvc -f win32 [AllOptions]


although this does not solve anything, only if explicitly specify the path, there may not be an error

@MVoz
Copy link
Contributor

MVoz commented Jul 3, 2020

other error

Spoiler

Creating library E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\lib\x64\hogweed.lib and object E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\lib\x64\hogweed.exp

     4>dsa-sign.obj : error LNK2019: unresolved external symbol mpz_powm_sec referenced in function nettle_dsa_sign [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-blind.obj : error LNK2001: unresolved external symbol mpz_powm_sec [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_mul_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_mul_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_mul referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_mul [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_div_r_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_div_r_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_div_r referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_div_r [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_powm_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_powm_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_powm referenced in function sec_powm [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_powm [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_add_1_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_add_1 referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2019: unresolved external symbol mpn_sec_invert_itch referenced in function rsa_sec_blind [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]
     4>rsa-sign-tr.obj : error LNK2019: unresolved external symbol mpn_sec_invert referenced in function rsa_sec_blind 

E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\bin\x64\hogweed.dll : fatal error LNK1120: 11 unresolved externals [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]

@MVoz
Copy link
Contributor

MVoz commented Jul 3, 2020

vcpkg remove gmp --triplet x64-windows

vcpkg install gmp --triplet x64-windows

vcpkg install nettle --triplet x64-windows

all ok

@MVoz
Copy link
Contributor

MVoz commented Jul 3, 2020

@Neumann-A

have a question for *. pc

gmp.txt
gmpxx.txt
libhogweed.txt
nettle.txt

example 1. libhogweed.pc -> 2. nettle.pc

-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/lib/pkgconfig/libhogweed.pc
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/lib/pkgconfig/nettle.pc
-- Fixing pkgconfig - debug
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/debug/lib/pkgconfig/libhogweed.pc
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/debug/lib/pkgconfig/nettle.pc
  1. double space hogweedd <-> nettled
    Libs: -L${libdir} -lhogweedd -lnettled
  2. separator ;
    Libs: -L${libdir} -lnettled;-lgmpd

@ghost
Copy link
Author

ghost commented Jul 3, 2020

@voskrese I tried making changes to the <vcpkg-root>\installed\x64-windows\shared\vs-yasm\yasm.props

  1. Deleted "$(YasmPath)"yasm.exe -> yasm.exe failed (yasm.exe is in path a link to yasm.exe in VSTools folder)
  2. Changed to include absolute path "$(YasmPath)"yasm.exe -> D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\yasm.exe
  3. set YASMPATH=D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\ and include it in vcpkg env passthrough
  4. I tried many other ways this and that spent most of the morning on this... :(

All seem to fail with about the same issue with YASM.

I will just figure something else out, especially since @JackBoosY and others can not repro it on their instances :(

@MVoz
Copy link
Contributor

MVoz commented Jul 3, 2020

test?

<YasmPath Condition= "'$(YASMPATH)' == ''">D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\</YasmPath>

@ghost
Copy link
Author

ghost commented Jul 3, 2020

Ok made change from:
<PropertyGroup>
<YasmPath Condition= "'$(YASMPATH)' == ''"></YasmPath>
</PropertyGroup>
to:
<PropertyGroup>
<YasmPath Condition= "'$(YASMPATH)' == ''">D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\</YasmPath>
</PropertyGroup>

Now it states no input file :(
_YASM:
x86_64\aes-decrypt-internal-2-fat.s
cmd.exe /D /C "S:\TEMP\vs2019.test\tmp14a20c86af284625a141ba926d321515.cmd"
"D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\"yasm.exe -Xvc -f win64 -o "D:\Git\vs2019.test\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\obj\ReleaseDLL\x64\libnettle\aes-decrypt-internal-2-fat.obj" -rraw -pgas "D:\Git\vs2019.test\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\x86_64\aes-decrypt-internal-2-fat.s"
yasm: No input files specified

I think I take a break on this one for a while, I am getting burnt out on it. I will look fresh later on.
Thank you very much for help so far - I am learning some new things, I appreciate the lessons. :)

@Neumann-A
Copy link
Contributor

@voskrese

separator ;

Ah thats an error in the portfile. set(LIBS -lnettle -lgmp) should probably be set(LIBS "-lnettle -lgmp")

double space hogweedd <-> nettled

doesn't matter. I also do not patch anything of this stuff (just configure it.)

@Sibras
Copy link
Contributor

Sibras commented Jul 4, 2020

FYI The 1.3.0 release of YASM has several bugs that cause segfaults and memory leaks when using the gas parser and COFF outputs. This is what is used by nettle and some other ports. So you may want to update the yasm port to instead pull in yasm master instead of the old 1.3.0 tag. This will fix some known issues, as to whether it fixes this issue i dont know.

@MVoz
Copy link
Contributor

MVoz commented Jul 4, 2020

@LilyWangL
https://github.com/microsoft/vcpkg/tree/master/ports/yasm

The 1.3.0 release of YASM has several bugs that cause segfaults and memory leaks

update + install tools ?

@MVoz
Copy link
Contributor

MVoz commented Jul 4, 2020

@Neumann-A

set(LIBS "-lnettle -lgmp")

yes, this corrected the situation
however, I don't know if the separator ; will be the cause of the error in the library search

@ghost
Copy link
Author

ghost commented Jul 4, 2020

@voskrese
cc: @JackBoosY @LilyWangL @Neumann-A @Sibras
Ok - I finally got nettle installed - here is what I did:

  1. Created a new CONTROL and portfile.cmake for yasm
  • CONTROL file change was replacing the version 1.3.0 with the lastest commit date at github.com/yasm 2020-05-04
  • portfile.cmake changed the REF to latest commit id and modified the SHA512 to match.
    vcpkg_from_github(
    OUT_SOURCE_PATH SOURCE_PATH
    REPO yasm/yasm
    REF c9db6d7
    SHA512 cbc083f80e61b898ee1eee99baa2c684fb5c928c8ceaa2dcd2e72e49fae8a1a86432df59e509abdec62343d989603440fa68b7996a25620dbc4303de0e295d02
    HEAD_REF master
    )
  1. Using Microsoft Windows [Version 10.0.19041.331]
    Visual Studio 2019 Version: 16.6.3 VS Tools Version: 14.26.28801
    .\vcpkg.exe install yasm[core]:x64-windows-static

  2. Copied the created 64-bit static yasm.exe to <vcpkg-root>\downloads\tools\yasm\yasm-1.3.0 replacing the 32-bit downloaded version by vcpkg.

  3. .\vcpkg.exe install nettle[core]:x64-windows - installed fine after making the one and only change (mention in 5 below) to gmp and nettle portfile.cmake file.

  4. I also had an issue with both gmp and nettle, where I had to change the following line:

    • from file(RENAME "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/include" "${CURRENT_PACKAGES_DIR}/include")
    • to file(INSTALL "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/include" DESTINATION "${CURRENT_PACKAGES_DIR}")
    • as it tells me trying to rename <vcpkg-root>\<port>\<triplet>-rel\<src-path>\msvc\include to
      <vcpkg-root>\packages\<port>_<triplet>\include already exists
    • but in reality it appears it is trying to rename it to <vcpkg-root>\installed\<triplet>\include\<port>
    • in essence I am not sure this is the correct use of cmake RENAME since the include directory/folder already exists and the portfile.cmake should have an cmake INSTALL instead? or could it be cmake version differences as I use cmake 3.17.3?
  5. I have noticed others getting similar errors in some of the issue posts mentioning rename failures and I am wondering if this is the same issue?

@MVoz
Copy link
Contributor

MVoz commented Jul 5, 2020

@Neumann-A

tested in
http://www.tortall.net/projects/yasm/releases/yasm-1.3.0-win64.exe
http://www.tortall.net/projects/yasm/releases/yasm-1.3.0-win32.exe

yasm x86 + props vs = error build x64


moy patch

vcpkg_find_acquire_program.cmake

...
    if(VCPKG_TARGET_ARCHITECTURE STREQUAL x86)
        set(URL "https://www.tortall.net/projects/yasm/snapshots/v1.3.0.6.g1962/yasm-1.3.0.6.g1962.exe")
        set(ARCHIVE "yasm-1.3.0.6-win32.exe")
        set(_vfa_RENAME "yasm.exe")
        set(NOEXTRACT ON)
        set(HASH c1945669d983b632a10c5ff31e86d6ecbff143c3d8b2c433c0d3d18f84356d2b351f71ac05fd44e5403651b00c31db0d14615d7f9a6ecce5750438d37105c55b)
    else(VCPKG_TARGET_ARCHITECTURE STREQUAL x64)
        set(URL "https://github.com/yasm/yasm/releases/download/v1.3.0/yasm-1.3.0-win64.exe")
        set(ARCHIVE "yasm-1.3.0-win64.exe")
        set(_vfa_RENAME "yasm.exe")
        set(NOEXTRACT ON)
        set(HASH 73dfd4ccf93972bb6e4794c071b712de0dbffe6d10345fd9d0b0a9c2472d87cd06f6ac32770af9ba2abb715ad0f80e2a55cf02284f44627dc5e303d66065336f)
    endif()
    get_filename_component(YASM_DIR "${PATHS}/${SUBDIR}" DIRECTORY)
    vcpkg_add_to_path(PREPEND "${YASM_DIR}")
    set(YASMPATH "${YASM_DIR}")
...

portfile.cmake

vcpkg_find_acquire_program(YASM) #<<-- can leave only one line

#    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
#    vcpkg_add_to_path("${YASM_DIR}")

#          /p:UseEnv=True

@MVoz
Copy link
Contributor

MVoz commented Jul 5, 2020

OR VS PROPS


portfile.cmake

    vcpkg_find_acquire_program(YASM)
    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
    vcpkg_add_to_path("${YASM_DIR}")
...
        OPTIONS
#          /p:UseEnv=True
+          "/p:YASMPATH=${YASM_DIR}/"

OR VS PROPS


portfile.cmake

    vcpkg_find_acquire_program(YASM)
    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
    vcpkg_add_to_path("${YASM_DIR}")
...
#        OPTIONS
#          /p:UseEnv=True

vcpkg_install_msbuild.cmake

...
    if(NOT DEFINED _csc_YASMPATH)
        set(_csc_YASMPATH ${YASM_DIR})
    endif()
...
...
    if(DEFINED _csc_YASMPATH)
		list(APPEND _csc_OPTIONS
		    "/p:YASMPATH=${YASM_DIR}/"
			)
    endif()

@JackBoosY
Copy link
Contributor

Related: #10613.

@JackBoosY
Copy link
Contributor

@voskrese Sadly, none of the solutions you provide can be used in Visual Studio 2017.

@Neumann-A
Copy link
Contributor

Hmm why not? I thought the solution would be to just download the correct architecture build of yasm? So just changing vcpkg_find_acquire_program

@JackBoosY
Copy link
Contributor

After changing the yasm binary, seems work.
I'm fixing this issue now.

@JackBoosY JackBoosY added category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly and removed category:port-bug The issue is with a library, which is something the port should already support labels Jul 8, 2020
@MVoz
Copy link
Contributor

MVoz commented Jul 8, 2020

Sadly, none of the solutions you provide can be used in Visual Studio 2017.

this case is voluntary to use or not, my business is to offer
and it works on MSVC 2013 b MSVC 2015
on account of 2017, I do not know, do not use

@ghost
Copy link
Author

ghost commented Jul 14, 2020

I close, I no longer use product.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants