west signing seems to be broken on windows #31876

thedjnK · 2021-02-02T15:27:20Z

Describe the bug
Trying to build a signed image, after running ninja it fails at the signing step with an error that %1 is not a valid win32 application and no signed image is created

To Reproduce
Steps to reproduce the behavior:

Build project with signing enabled and setup (I used philospher example, I used ninja menuconfig to enable mcuboot support, add the sample key file and enable outputting a confirmed image)
Build with ninja
Error shown is:

    [1/1] Linking C executable zephyr\zephyr.elf
    FAILED: zephyr/zephyr.elf zephyr/zephyr.hex zephyr/zephyr.bin zephyr/zephyr.lst zephyr/zephyr.stat zephyr/zephyr.signed.bin zephyr/zephyr.signed.confirmed.bin zephyr/zephyr.signed.hex zephyr/zephyr.signed.confirmed.hex 
    cmd.exe /C "cd . && C:\gcc-arm\bin\arm-none-eabi-gcc.exe   zephyr/CMakeFiles/zephyr_final.dir/misc/empty_file.c.obj zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj -o zephyr\zephyr.elf  -Wl,-T  zephyr/linker_pass_final.cmd  -Wl,-Map=C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr_final.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/serial/libdrivers__serial.a  modules/hal_nordic/libmodules__hal_nordic.a  modules/mcuboot/lib..__bootloader__mcuboot__boot__bootutil__zephyr.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/gcc-arm/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v8-m.main/nofp"  -LC:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cmd.exe /C "cd /D C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu\zephyr && "C:\Program Files\CMake\bin\cmake.exe" -E rename zephyr_final.map zephyr.map && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=ihex --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.hex && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=binary --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.bin && C:\gcc-arm\bin\arm-none-eabi-objdump.exe -d -S zephyr.elf > zephyr.lst && C:\gcc-arm\bin\arm-none-eabi-readelf.exe -e zephyr.elf > zephyr.stat && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem --pad --confirm""
    Traceback (most recent call last):
      File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "C:\Python37\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 806, in <module>
        main()
      File "C:\Python37\lib\site-packages\west\app\main.py", line 779, in main
        app.run(argv or sys.argv[1:])
      File "C:\Python37\lib\site-packages\west\app\main.py", line 106, in run
        self.run_command(argv)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 338, in run_command
        self.run_extension(args.command, argv)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 408, in run_extension
        command.run(args, unknown, self.topdir, manifest=self.manifest)
      File "C:\Python37\lib\site-packages\west\commands.py", line 129, in run
        self.do_run(args, unknown)
      File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 187, in do_run
        signer.sign(self, build_dir, bcfg, formats)
      File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 278, in sign
        subprocess.check_call(sign_bin)
      File "C:\Python37\lib\subprocess.py", line 336, in check_call
        retcode = call(*popenargs, **kwargs)
      File "C:\Python37\lib\subprocess.py", line 317, in call
        with Popen(*popenargs, **kwargs) as p:
      File "C:\Python37\lib\subprocess.py", line 769, in __init__
        restore_signals, start_new_session)
      File "C:\Python37\lib\subprocess.py", line 1172, in _execute_child
        startupinfo)
    OSError: [WinError 193] %1 is not a valid Win32 application
    ninja: build stopped: subcommand failed.

Expected behavior
The application to be signed

Impact
Showstopper

Environment (please complete the following information):

OS: Windows 7 64-bit
Toolchain: GCC ARM 9
Commit: 8925af9
West: 0.9.0
Python: 3.7

The text was updated successfully, but these errors were encountered:

carlescufi · 2021-02-02T18:13:28Z

@thedjnK what exact commands did you run? Could you please document those here so that we can try to reproduce?

mbolivar-nordic · 2021-02-02T18:13:56Z

@thedjnK I think what's going on here is that you neither have imgtool installed via pip3 nor have the registry setup to run .py files as executables.

Without imgtool installed from pip3, west sign will run the imgtool.py script in the mcuboot repository. If Windows is not configured to run the correct interpreter for a .py file, you'll get this error.

python.org says that the default installer should make the right file associations: https://docs.python.org/3.9/using/windows.html#from-file-associations

However, that's not working on my Windows setup, either.

I'll think about how to fix this, but in the meantime, you have these workarounds:

pip3 install imgtool
provide --tool-path and point it at a .bat script that runs imgtool.py in your mcuboot repo
put an imgtool script in your PATH that runs imgtool.py in your mcuboot repo

mbolivar-nordic · 2021-02-02T18:14:07Z

@nashif consider changing to priority low based on the above diagnosis

carlescufi · 2021-02-02T18:14:26Z

@nashif consider changing to priority low based on the above diagnosis

Done now.

thedjnK · 2021-02-02T18:27:42Z

@mbolivar-nordic I've checked and python files are associated with python 3.7, if I create an empty .py file and run it, I get the python console open and close very quickly. I didn't have imgtool installed via pip as I assumed it would use the bundled version with zephyr as it used to do in zephyr 1.x, so I've installed imgtool via pip and I'm still getting the same result:

C:\pure_zephyr\zephyr>pip install imgtool
Collecting imgtool
  Downloading https://files.pythonhosted.org/packages/6f/bb/712f20549ab5a82571009eb4fef9b6f0b71a4685fa28bfaa9c1cbabd6d17/imgtool-1.7.1-py3-none-any.whl
Requirement already satisfied: cbor>=1.0.0 in c:\python37\lib\site-packages (from imgtool) (1.0.0)
Requirement already satisfied: click in c:\python37\lib\site-packages (from imgtool) (6.6)
Requirement already satisfied: intelhex>=2.2.1 in c:\python37\lib\site-packages (from imgtool) (2.2.1)
Requirement already satisfied: cryptography>=2.4.2 in c:\python37\lib\site-packages (from imgtool) (2.9.2)
Requirement already satisfied: six>=1.4.1 in c:\python37\lib\site-packages (from cryptography>=2.4.2->imgtool) (1.12.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in c:\python37\lib\site-packages (from cryptography>=2.4.2->imgtool) (1.14.0)
Requirement already satisfied: pycparser in c:\python37\lib\site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.4.2->imgtool) (2.20)
Installing collected packages: imgtool
Successfully installed imgtool-1.7.1
WARNING: You are using pip version 19.2.2, however version 21.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

C:\pure_zephyr\zephyr>cd C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu

C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu>ninja
[1/1] Linking C executable zephyr\zephyr.elf
FAILED: zephyr/zephyr.elf zephyr/zephyr.hex zephyr/zephyr.bin zephyr/zephyr.lst
zephyr/zephyr.stat zephyr/zephyr.signed.bin zephyr/zephyr.signed.confirmed.bin zephyr/zephyr.signed.hex zephyr/zephyr.signed.confirmed.hex
cmd.exe /C "cd . && C:\gcc-arm\bin\arm-none-eabi-gcc.exe   zephyr/CMakeFiles/zephyr_final.dir/misc/empty_file.c.obj zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj -o zephyr\zephyr.elf  -Wl,-T  zephyr/linker_pass_final.cmd  -Wl,-Map=C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr_final.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/serial/libdrivers__serial.a  modules/hal_nordic/libmodules__hal_nordic.a  modules/mcuboot/lib..__bootloader__mcuboot__boot__bootutil__zephyr.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/gcc-arm/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v8-m.main/nofp"  -LC:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cmd.exe /C "cd /D C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu\zephyr && "C:\Program Files\CMake\bin\cmake.exe" -E rename zephyr_final.map zephyr.map && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=ihex --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.hex && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=binary --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.bin && C:\gcc-arm\bin\arm-none-eabi-objdump.exe -d -S zephyr.elf > zephyr.lst && C:\gcc-arm\bin\arm-none-eabi-readelf.exe -e zephyr.elf > zephyr.stat && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem --pad --confirm""
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 806, in <module>
    main()
  File "C:\Python37\lib\site-packages\west\app\main.py", line 779, in main
    app.run(argv or sys.argv[1:])
  File "C:\Python37\lib\site-packages\west\app\main.py", line 106, in run
    self.run_command(argv)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 338, in run_command
    self.run_extension(args.command, argv)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 408, in run_extension
    command.run(args, unknown, self.topdir, manifest=self.manifest)
  File "C:\Python37\lib\site-packages\west\commands.py", line 129, in run
    self.do_run(args, unknown)
  File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 187, in do_run
    signer.sign(self, build_dir, bcfg, formats)
  File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 278, in sign
    subprocess.check_call(sign_bin)
  File "C:\Python37\lib\subprocess.py", line 336, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Python37\lib\subprocess.py", line 317, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Python37\lib\subprocess.py", line 769, in __init__
    restore_signals, start_new_session)
  File "C:\Python37\lib\subprocess.py", line 1172, in _execute_child
    startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application
ninja: build stopped: subcommand failed.

C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu>imgtool version
1.7.1

mbolivar-nordic · 2021-02-02T18:28:39Z

Did you try with a pristine build? I can no longer reproduce the error on my windows machine after installing imgtool.

thedjnK · 2021-02-02T18:30:31Z

Exact commands were:

mkdir build
cd build
cmake -GNinja -DBOARD=<our_board> ..
ninja menuconfig
mcuboot support was selected and confirmed image was selected, the mcuboot key file was also added and configuration was saved
ninja
build failed with above error

mbolivar-nordic · 2021-02-02T18:31:58Z

I'm building hello_world successfully with this command after running pip3 install imgtool:

west build -b nrf52dk_nrf52832 -s zephyr/samples/hello_world -d build-hello-signed -- -DCONFIG_BOOTLOADER_MCUBOOT=y -DCONFIG_MCUBOOT_SIGNATURE_KEY_FILE=\"bootloader/mcuboot/root-rsa-2048.pem\"

The above comes from the west sign docs here:

https://docs.zephyrproject.org/latest/guides/west/sign.html

Can you try that, to see if perhaps it's a Kconfig issue?

thedjnK · 2021-02-02T18:32:21Z

@mbolivar-nordic Ah yes, after clearing the build directory and re-generating it, it's now building and signing, thank you

We can't trust that a python file is executable on Windows, regardless of what the mode bits say. When we find that imgtool is a .py file, run 'python imgtool.py' instead of 'imgtool.py' on that platform. Fixes: zephyrproject-rtos#31876 Signed-off-by: Martí Bolívar <[email protected]>

mbolivar-nordic · 2021-02-02T20:53:59Z

@thedjnK could you please test that #31890 fixes your issue even with imgtool uninstalled (pip3 uninstall imgtool) with a pristine build directory?

thedjnK · 2021-02-02T21:49:11Z

@mbolivar-nordic tested and that patch resolves the issue, thank you

mbolivar-nordic · 2021-02-02T22:00:54Z

Thanks for testing!

We can't trust that a python file is executable on Windows, regardless of what the mode bits say. When we find that imgtool is a .py file, run 'python imgtool.py' instead of 'imgtool.py' on that platform. Fixes: #31876 Signed-off-by: Martí Bolívar <[email protected]>

thedjnK added the bug The issue is a bug, or the PR is fixing a bug label Feb 2, 2021

nashif assigned mbolivar-nordic Feb 2, 2021

nashif added area: West West utility priority: medium Medium impact/importance bug labels Feb 2, 2021

carlescufi added priority: low Low impact/importance bug and removed priority: medium Medium impact/importance bug labels Feb 2, 2021

mbolivar-nordic mentioned this issue Feb 2, 2021

west sign: don't try to execute imgtool.py on windows #31890

Merged

nashif closed this as completed in #31890 Feb 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

west signing seems to be broken on windows #31876

west signing seems to be broken on windows #31876

thedjnK commented Feb 2, 2021

carlescufi commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021 •

edited

Loading

mbolivar-nordic commented Feb 2, 2021

carlescufi commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

west signing seems to be broken on windows #31876

west signing seems to be broken on windows #31876

Comments

thedjnK commented Feb 2, 2021

carlescufi commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021 • edited Loading

mbolivar-nordic commented Feb 2, 2021

carlescufi commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

thedjnK commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021

mbolivar-nordic commented Feb 2, 2021 •

edited

Loading