Native Windows build #38872

tobiasdiez · 2024-10-28T10:54:34Z

The goal of this PR is to provide a native Windows build of Sage using MSVC (using the new Meson build system).

Instructions on how to test are under the Windows tab at https://doc-pr-38872--sagemath.netlify.app/html/en/installation/meson.
Afterwards, you should be able to import some sage modules in a normal python install (but most modules actually don't properly import due to various errors that will be fixed peu a peu in follow-up PRs)

State of Windows support of dependencies

m4rie: upstream (https://github.com/malb/m4rie) doesn't provide win build
ec: not sure if upstream (https://github.com/JohnCremona/eclib) provide win build
ecm: upstream (https://gitlab.inria.fr/zimmerma/ecm) doesn't provide native win build (only Cygwin)
ntl: upstream provides win build (kind of), but it's not yet available on conda (Build for windows with CMake conda-forge/ntl-feedstock#16)
maxima/ecl: upstream provides win build, but it's not yet available on conda (Windows build conda-forge/maxima-feedstock#38)
GAP: only cygiwn build exists Thoughts for a native (well, mingw) Window version of GAP gap-system/gap#4157
pari: upstream supports building on Windows using mingw, but it's not yet available on conda (Windows version conda-forge/pari-feedstock#45)
cypari2: no windows support yet merge cypari and cypari2 cypari2#19
lrcalc
pplpy
primecountpy
fpylll
rw
cliquer
lcalc
ecl
braiding
gc
homfly
iml
brial_groebner

Conda Issues:

Meson cannot find libraries without pkg-config files.
- Workaround: Check that set LIB includes the conda env (e.g. ...\.conda\envs\sage-dev\Library\lib). If not, run conda deactivate followed by conda activate sage-dev and then check again.
- Alternatively, set LIB=%CONDA_PREFIX%\Library\lib;%LIB%
- Possible upstream improvement: the compilers conda package should set LIB
If you don't have the debug version of Python installed, you may get "LINK : fatal error LNK1104: cannot open file 'python311_d.lib'"
- Workaround: meson setup builddir -Dbuildtype=release
- Alternatively, in <path to conda>\sage-dev\include\pyconfig.h uncomment: //# define Py_DEBUG and change python311_d.lib to python311.lib
- Possible upstream improvement: conda should either ship a debug version of python or a correct pyconfig.h
pkgconfig is wrong conda-forge/gsl-feedstock#63
- Workaround: Fallback to find_library in meson
When installingc-compiler on Windows, VS 2022 Build Tools init file is not installed conda-forge/compilers-feedstock#66
- Workaround: Run everything in "VS x64 Native Tools Command Prompt" (for 64bit) or "Developer Command Prompt for VS2022 (or 2019)" (for 32bit)
Doesn't include a pkg-config file (on Windows) conda-forge/libgd-feedstock#55
- Workaround: Fallback to find_library in meson
Linker issue LNK2019: unresolved external symbol on Windows conda-forge/m4ri-feedstock#15
- Workaround: Don't try to use m4ri on windows
winpthreads-related issue when compiling against flint conda-forge/libflint-feedstock#38
- Workaround: Don't try to use flint on windows

Upstream issues

Singular

Flint:

Fix name conflicts with small macro from rpcndr flintlib/flint#2100
Using winpthreads with python leads to confliciting pid_t mingw-w64/mingw-w64#65 (indirectly since pthread is flint dependency). Would be fixed by winpthreads-related issue when compiling against flint conda-forge/libflint-feedstock#38.
-> Workaround: Change typedef __int64 pid_t; to typedef int pid_t; in .conda\envs\sage-dev\Library\include\pthread_compat.h
Replace inline assembly with function prototypes in sanity check flintlib/flint#2101:
-> Workaround: disable compilation of fraction_field_FpT for now

Cysignals:

📝 Checklist

The title is concise and informative.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation and checked the documentation preview.

⌛ Dependencies

github-actions · 2024-10-28T11:43:25Z

Documentation preview for this PR (built with commit 0445ffb; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

../src/sage/symbolic/ginac/upoly-ginac.cpp(219): error C2440: '<function-style-cast>': cannot convert from 'size_t' to 'GiNaC::numeric' ../src/sage/symbolic/ginac/upoly-ginac.cpp(219): note: 'GiNaC::numeric::numeric': ambiguous call to overloaded function

../src/sage/symbolic/ginac/archive.cpp(584): error C2039: 'mem_fun_ref': is not a member of 'std' C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\sstream(19): note: see declaration of 'std' ../src/sage/symbolic/ginac/archive.cpp(584): error C3861: 'mem_fun_ref': identifier not found ../src/sage/symbolic/ginac/archive.cpp(584): error C2672: 'for_each': no matching overloaded function found

C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\xkeycheck.h(341): warning C4005: 'register': macro redefinition ../src/sage/symbolic/ginac/constant.cpp(23): note: see previous definition of 'register' C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\xkeycheck.h(343): fatal error C1189: #error: The C++ Standard Library forbids macroizing the keyword "register". Enable warning C4005 to find the forbidden define.

@ginac

ginac_lst.cpp.obj : error LNK2005: "protected: static void const * __cdecl GiNaC::container<class std::list>::get_tinfo(void)" (?get_tinfo@?$container@Vlist@std@@@ginac@@KAPEBXXZ) already defined in meson-generated_src_sage_symbolic_expression.pyx.cpp.obj

Some of them should perhaps be reverted

`os.uname` is not available on Windows

tobiasdiez · 2024-12-05T02:36:49Z

@saraedum Thanks for the feedback. Cysignals is now using meson and builds nicely on Windows. NTL is still not available on conda (as far as I know), but we can use the flexibility of meson to just not build the parts of sage that make use of it (a similar strategy is employed for various other dependencies which don't provide windows build / or are not installable from conda yet).

This PR is now in a working stage, at least to the extend that it should compile nicely on Windows. Instructions on how to test are under the Windows tab at https://doc-pr-38872--sagemath.netlify.app/html/en/installation/meson. If you find the time to test it on your system, this would be awesome. After the next beta (that includes #37447) I'll cleanup the code a bit more and make the PR ready for a proper review.

Turns out `mtx.lib` exists on Windows as "Microsoft Transaction Server". Thus, we also check for the existence of the header `meataxe.h`.

isuruf · 2024-12-09T13:25:50Z

cysignals and gmpy2 issues should be fixed now

…eson-win

tobiasdiez · 2024-12-17T07:49:45Z

cysignals and gmpy2 issues should be fixed now

They are indeed, thanks a lot!

…eson-win

soehms · 2025-01-08T17:00:29Z

This PR is now in a working stage, at least to the extend that it should compile nicely on Windows. Instructions on how to test are under the Windows tab at https://doc-pr-38872--sagemath.netlify.app/html/en/installation/meson. If you find the time to test it on your system, this would be awesome. After the next beta (that includes #37447) I'll cleanup the code a bit more and make the PR ready for a proper review.

Testreport on Windows 11 pro

I tried to follow your instructions on the documentation page. Since I'm not really familiar with the build system on Windows, it wasn't easy. First of all, it looks like the "Windows" tab is almost a clone of the other tabs with very few Windows-specific changes. Even your example of the "VS x64 Native Tools Command Prompt" doesn't look like what it looked like in my test:

**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.12.3
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Program Files\Microsoft Visual Studio\2022\Community>

I ran the test on Windows 11 Pro, freshly installed a few months ago. None of the required prerequisites were present on that system. Here I give a summary of what I had to do to get the test working (hoping it helps someone else starting at a similar point as me):

I downloaded and ran VisualStudioSetup. I checked all the boxes relevant to C/C++ and Python development (and a few more that may not be required but I wasn't sure about).
I downloaded and installed Git-Bash, selecting all the default options except for the question about the terminal I want Git to be available on (here I selected the option for all terminal types).
I downloaded miniforge and installed it under my home path using Powershell as explained in this tutorial.

After that, I started the VS x64 Native Tools prompt (I suspect that's essentially a regular cmd terminal with the environment set by "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsall.bat"). I changed the directory to my home path, cloned the Sage repository into a subfolder, and checked out your branch. Then I activated the mamba environment by running .\miniforge3\Scripts\activate.

Finally, I was able to follow your instructions:

(sage-dev) C:\Users\sebastian\sage>pip install gmpy2 cysignals
Requirement already satisfied: gmpy2 in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (2.1.5)
Requirement already satisfied: cysignals in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (1.12.0)

(sage-dev) C:\Users\sebastian\sage>set LIB=%CONDA_PREFIX%\Library\lib;%LIB%

(sage-dev) C:\Users\sebastian\sage>pip install --no-build-isolation --editable .
Obtaining file:///C:/Users/sebastian/sage
  Checking if build backend supports build_editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: six>=1.15.0 in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (from sagemath==10.6b2) (1.17.0)
Requirement already satisfied: conway-polynomials>=0.8 in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages(from sagemath==10.6b2) (0.10)
Requirement already satisfied: cython!=3.0.3,>=3.0 in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (from sagemath==10.6b2) (3.0.11)
Requirement already satisfied: gmpy2~=2.1.b999 in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (from sagemath==10.6b2) (2.1.5)
...
Requirement already satisfied: pure_eval in c:\users\sebastian\miniforge3\envs\sage-dev\lib\site-packages (from stack_data->ipython>=7.13.0->sagemath==10.6b2) (0.2.3)
Building wheels for collected packages: sagemath
  Building editable for sagemath (pyproject.toml) ... done
  Created wheel for sagemath: filename=sagemath-10.6b2-cp311-cp311-win_amd64.whl size=16759 sha256=53b80ef9857cb142c2919a88ff11f1f1384b09e6f91fa52263842d981871d19c
  Stored in directory: C:\Users\sebastian\AppData\Local\Temp\pip-ephem-wheel-cache-bxq6ke4c\wheels\6c\82\8f\5ff77f17cf38bb97b46a696d2aefe5aa0fff1e75fe9cde8b7d
Successfully built sagemath
Installing collected packages: sagemath
Successfully installed sagemath-10.6b2

Now I'm not quite sure how to use Sage? There was nothing in the terminal environment to start Sage by name. Then I tried to use Sage in an IPython session by importing its global names...

sage-dev) C:\Users\sebastian\sage>IPython
Python 3.11.11 | packaged by conda-forge | (main, Dec  5 2024, 14:06:23) [MSC v.1942 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.30.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from sage.rings.all import *
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 from sage.rings.all import *

File ~\sage\src\sage\rings\all.py:57
     54 from sage.rings.finite_rings.all import *
     56 # Number field
---> 57 from sage.rings.number_field.all import *
     59 # Function field
     60 from sage.rings.function_field.all import *

File ~\sage\src\sage\rings\number_field\all.py:2
----> 2 from sage.rings.number_field.number_field import (NumberField, NumberFieldTower, CyclotomicField, QuadraticField,
      3                                                   is_fundamental_discriminant, is_real_place)
      4 from sage.rings.number_field.number_field_element import NumberFieldElement
      6 from sage.rings.number_field.order import EquationOrder, GaussianIntegers, EisensteinIntegers
    198 from sage.modules.module import Module

File ~\sage\src\sage\rings\number_field\number_field.py:118
    116 from sage.categories.homset import Hom
    117 from sage.categories.sets_cat import Sets
--> 118 from sage.modules.free_module import VectorSpace
    119 from sage.modules.free_module_element import vector
    120 from sage.rings.real_mpfr import RR

File ~\sage\src\sage\modules\free_module.py:197
    195 from sage.misc.lazy_import import LazyImport
    196 from sage.misc.randstate import current_randstate
--> 197 from sage.modules import free_module_element
    198 from sage.modules.module import Module
    199 from sage.rings.finite_rings.finite_field_base import FiniteField

ImportError: cannot import name 'free_module_element' from 'sage.modules' (C:\Users\sebastian\sage\src\sage\modules\__init__.py)

In [2]:

In [2]: from sage.rings.integer_ring import ZZ

In [3]: ZZ
Out[3]: Integer Ring

tobiasdiez and others added 12 commits October 23, 2024 01:15

Merge remote-tracking branch 'origin/conda-improv' into meson-win

89d7cc5

Generate conda environment files for windows

22c75a5

Mark a few dependencies as optional on windows

0769f31

Merge remote-tracking branch 'upstream/develop' into meson-win

149c569

Minor improvements to meson build files

6cde21b

Meson: mark some dependencies as optional on Windows

b10e365

Require cysignals again

6846003

Meson: Improve handling of dependencies

e950d9d

Meson: add singular (factory) as subproject

6c70e45

Meson: a few more minor revisions

a5d70af

Meson: improve handling of dependencies

526eae4

Exclude a few files from compilation due to errors on windows

8fc3cb6

tobiasdiez added 14 commits October 28, 2024 21:32

fix meson build

8f705dc

fix singular factory declaration in meson

1861ec0

Provide wrappers srandom and random for Windows

bed071f

Replace division by zero with +-inf

a5d85ce

Fix Ginac cast error

0e70b7a

../src/sage/symbolic/ginac/upoly-ginac.cpp(219): error C2440: '<function-style-cast>': cannot convert from 'size_t' to 'GiNaC::numeric' ../src/sage/symbolic/ginac/upoly-ginac.cpp(219): note: 'GiNaC::numeric::numeric': ambiguous call to overloaded function

Replace unistd.h by windows.h on Windows

5062be7

Narrow down imports

a9e0cb1

Some of them should perhaps be reverted

Replace os.uname by more universal platform.system

d8b6466

`os.uname` is not available on Windows

hide some errors for now

5a885aa

Fix syntax error

6014f7e

Catch missing singular dependency

bbd8b48

This was referenced Nov 3, 2024

Meson: minor revision #38910

Merged

Replace division by zero with +-inf #38911

Merged

Replace deprecated/removed mem_fun_ref #38912

Merged

Add fortran compiler as host requirement

6a498f7

tobiasdiez mentioned this pull request Nov 30, 2024

Add initial support for building with mingw #35796

Open

5 tasks

tobiasdiez added 5 commits December 2, 2024 16:13

No editable on install on linux

baeb2ae

Merge remote-tracking branch 'origin/conda-grayskull' into meson-win

0892c0d

Merge remote-tracking branch 'upstream/develop' into meson-win

2939535

Fix script

5bb211f

Add docs

8ee1c4a

tobiasdiez marked this pull request as ready for review December 5, 2024 02:33

github-actions bot added the s: needs review label Dec 5, 2024

tobiasdiez added 3 commits December 9, 2024 18:57

Merge remote-tracking branch 'origin/conda-grayskull' into meson-win

17e7910

Fix non-editable install on windows

bd63f49

Fix mtx/meataxe detection on Windows

e755a5a

Turns out `mtx.lib` exists on Windows as "Microsoft Transaction Server". Thus, we also check for the existence of the header `meataxe.h`.

tobiasdiez and others added 4 commits December 10, 2024 00:02

update win lock files to include cysignals and gmpy2

d4924b0

No longer need to install cysignals and gmpy2 manually

20e46c9

Merge branch 'ship-init' into meson-win

fc3e84c

Merge branch 'meson-win' of https://github.com/tobiasdiez/sage into m…

bb5b0fa

…eson-win

tobiasdiez and others added 6 commits December 23, 2024 05:25

Merge branch 'develop' into meson-win

4b29be4

Merge branch 'meson-win' of https://github.com/tobiasdiez/sage into m…

1686052

…eson-win

cleanup

21a896e

Fix linter

7877543

Merge remote-tracking branch 'upstream/develop' into meson-win

d4cc54e

fix sysconf import

0445ffb

tobiasdiez requested review from saraedum, dimpase, kwankyu and soehms January 5, 2025 10:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Native Windows build #38872

Native Windows build #38872

tobiasdiez commented Oct 28, 2024 •

edited

Loading

github-actions bot commented Oct 28, 2024 •

edited

Loading

tobiasdiez commented Dec 5, 2024 •

edited

Loading

isuruf commented Dec 9, 2024

tobiasdiez commented Dec 17, 2024

soehms commented Jan 8, 2025

Native Windows build #38872

Are you sure you want to change the base?

Native Windows build #38872

Conversation

tobiasdiez commented Oct 28, 2024 • edited Loading

State of Windows support of dependencies

Conda Issues:

Upstream issues

📝 Checklist

⌛ Dependencies

github-actions bot commented Oct 28, 2024 • edited Loading

tobiasdiez commented Dec 5, 2024 • edited Loading

isuruf commented Dec 9, 2024

tobiasdiez commented Dec 17, 2024

soehms commented Jan 8, 2025

Testreport on Windows 11 pro

tobiasdiez commented Oct 28, 2024 •

edited

Loading

github-actions bot commented Oct 28, 2024 •

edited

Loading

tobiasdiez commented Dec 5, 2024 •

edited

Loading