MinGW-W64 cross-compilation bug roundup

[taktoa]

Issue description

I'm going to use this issue as a dumping ground for any bugs I find while fixing stuff related to the MinGW-W64 cross-compilation feature of nixpkgs.

Bugs

• Upgraded MinGW-W64 to stable version 5.0.3 #24985 mingw-w64 should be upgraded to the latest version 5.0.2 5.0.3
  • I am currently working on a pull request for this. @angerman did this...somewhere.
• coreutils build is broken because attr needs sys/resource.h.
  • Since it doesn't make sense to cross-build attr for Windows (it's only needed for displaying extended file attributes in ls), we should just set attr = null when building with MinGW-W64.
  • I tried building (coreutils.override { attr = null; }) in vanilla (non-cross) nixpkgs and it worked, so attr really is an optional dependency here.
  • Cross-building coreutils for Windows is not particularly useful anyway, but since I've already figured out a fix for it I thought I should write it down somewhere, in case someone really does need it to build.
• libsigsegv fails to build with the following error:

  handler-win32.c:218:43: error: 'struct _CONTEXT' has no member named 'Esp'
  handler-win32.c:220:43: error: 'struct _CONTEXT' has no member named 'Eip'
  

• Get std::future working. See MinGW-W64 cross-compilation bug roundup #24984 (comment)

Other notes

If you currently use the MinGW-W64 cross-compilation support in nixpkgs, please let me know if there are any specific packages I should try to keep working.

cc @DavidEGrayson
cc @cleverca22

[DavidEGrayson]

• Get std::future working.

The last time I checked, the C++ std::future class was not working in the mingw-w64 toolchain from nixpkgs. There was a compile-time error due to the class being disabled. I think you need to build winpthreads and install it at the same as the mingw-crt, and then use --enable-threads=posix when building the final GCC toolchain. Then GCC's libstdc++ will use winpthreads to implement std::future and things like that. I got it working in nixcrpkgs if you want to take a look.

[domenkozar]

what's the status here? :)

[taktoa]

IIRC there have been some major changes to the Nix cross compilation framework by @Ericson2314, not sure how this is affected by that.

[Ericson2314]

Assuming we can speed up reviewing enough that less-effiencient development in parallel working, lets get everything in #30882 merged first, then all the non-mingw cross tests (https://hydra.nixos.org/jobset/nixpkgs/cross-trunk) should be passing. Then it will be easier to go after mingw stuff in isolation.

[Ericson2314]

@angerman any of this remains?

[angerman]

@Ericson2314 bumping mingw certainly. There some other strange linking issues I’m currently following. Not sure if this is only related to the Haskell builder or a broader issue.

[taktoa]

@angerman if you end up making a PR to bump the MinGW version, let me know so that I can close #24985 (I haven't had the time to work on this stuff lately, but that PR should be a good jumping-off point if you're going to bump the version)

[angerman]

Just for the record. I'm probably going to try and integrate most of #24985, and might take some inspiration from nixcrpkgs. However, my focus is to utilize the mingw-w64 toolchain to cross compile haskell.

[tex]

Hi, I am trying to cross compile my package which uses std c++11 features like std::mutex which seems to be supported with mingw w64 with posix threading model.
With current master nixpkgs cross compilation fails on error: 'mutex' in namespace 'std' does not name a type...

Any idea what needs to be done to get mingw w64 with posix threading model with nixpkgs?

[taktoa]

@tex I can't remember if I ever got that working, but I remember running into issues like that when working on this project, so you can search in that code for clues. Note that that code is written for the old version of nixpkgs cross-compilation, but the answer you seek should be agnostic to that.

[DavidEGrayson]

@tex I gave a quick summary of what I think needs to happen in my last comment (from two years ago): #24984 (comment)

[tex]

Thanks. So if I understand correctly, current nixpgs builds just one-stage gcc which leads to use of win32 threading model that doesn't support c++11.
Simply adding --with-libraries=winpthread to configureFlags in mingw-w64/default.nix and --enable-threads=posix doesn't help either because it tries to build it for build architecture, not host which leads to

libtool: link: x86_64-pc-mingw32-gcc -shared  src/.libs/libwinpthread_la-barrier.o src/.libs/libwinpthread_la-cond.o src/.libs/libwinpthread_la-misc.o src/.libs/libwinpthread_la-mutex.o src/.libs/libwinpthread_la-rwlock.o src/.libs/libwinpthread_la-spinlock.o src/.libs/libwinpthread_la-thread.o src/.libs/libwinpthread_la-ref.o src/.libs/libwinpthread_la-sem.o src/.libs/libwinpthread_la-sched.o src/.libs/libwinpthread_la-clock.o src/.libs/libwinpthread_la-nanosleep.o src/.libs/version.o   -L./fakelib  -g -O2 -no-pthread   -o .libs/libwinpthread-1.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libwinpthread.dll.a
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find dllcrt2.o: No such file or directory
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmingw32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmoldname
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmingwex
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmsvcrt
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -ladvapi32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lshell32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -luser32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lkernel32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmingw32
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmoldname
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmingwex
/nix/store/giwax7g45z90fb1lwvmbrl1db5bmaxb4-x86_64-pc-mingw32-binutils-2.31.1/bin/x86_64-pc-mingw32-ld: cannot find -lmsvcrt
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:680: libwinpthread.la] Error 1
make[4]: Leaving directory '/build/mingw-w64-v6.0.0/mingw-w64-libraries/winpthreads'

So two-stage is needed, first stage to build current mingw-w64 that will be used to build winpthreads which will be used in second stage.

Correct?

Is it feasable to do that with what currently is laid out in nixpgs (pkgs/os-specific/windows/...)?

I wish I knew more about nix syntax...

[angerman]

@tex, I've tried to get this working for us. About half a year ago in a bid to cross compile rocksdb, with the nixpkgs setup at that point, and I think @Ericson2314 agrees with me here, it would require to actually split gcc into various packages and take it apart. That would also solve some of the static gcc parts.

While I did manage to mutilate a nixpkgs configuration bad enough to have a gcc with winpthreads, I ended giving up on it as I could not produce a rocksdb with it that was sufficiently similar in behaviour to the one built with the microsoft toolchain. Thus we ended up with a rocksdb binary blob for windows.

If you dig deeper into the whole threading situation with ming32 and c++, it's not pretty. I'm thankful we haven't run into any other annoying c++ code, and thus did not have to revive the need.

[tex]

Hi @angerman, do you still have the nix configs for the gcc with pthreads so I could use them for my experiments?

[angerman]

@tex not sure how complete this is, but there are a few commits here: https://github.com/angerman/nixpkgs/tree/feature/gcc-ng, you may want to look for commits by me around the same time to that repo as well.

[Ericson2314]

With #73195 the C++ concurrency datatypes should now be available.

[leo60228]

SDL2 builds broken .la files, saying the application should look for a .a in /nix/store/xxx/lib instead of a .dll in /nix/store/xxx/bin.

[stale]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[AleXoundOS]

Currently, not a user of mingw-w64 cross-compilation on nix. But I plan to use it in the nearest future and don't know beforehand whether I will be affected by the issue.

[stale]

I marked this as stale due to inactivity. → More info

[ramirez7]

SDL2 builds broken .la files, saying the application should look for a .a in /nix/store/xxx/lib instead of a .dll in /nix/store/xxx/bin.

@leo60228 Do you know of a fix for this? I'm currently x-compiling SDL2 to mingwW64 (for use with the Haskell sdl2 bindings), and I'm getting errors that may be caused by this (Haskell build cannot find SDL2 and SDL2main)

[stale]

I marked this as stale due to inactivity. → More info

[Thesola10]

@ramirez7 I believe this can fix it, but the option should be bound to pkgsStatic.