add scipy

[wolfv]

It's far from working, though!

[DerThorsten]

todos:

• add pyodide LICENCE to patch folder
• put FORTRAN.py also in different folder st. this folder can have pyodides LICENCE

[wolfv]

After disabling pthreads in pypocketfft, things are loading (almost) correctly in the browser. Another needed manual fix is that _cpropack needs to come before _zpropack and I had to manually resort it in the python_data.js.

We still have some remaining issues:

• pythran is not working yet
• scipy.optimize - seems like the Fortran code cannot properly call the Python function
• sorting in empack (as mentioned above)

The following does not work currently (FORTRAN error calling function?).

import numpy as np
from scipy.optimize import minimize, rosen, rosen_der

def f(x):
    return x**2 + 10*np.sin(x)

print(minimize(f, x0=0))

[wolfv]

Pythran should be fixed after removing strip all here: #129
Will try now!

[wolfv]

I checked the pyodide binaries and there is a difference with ours for the propack stuff.

Ours:

[w@fedora _propack]$ wasm-nm _cpropack.cpython-310.so | grep bb
i bbcom_
[w@fedora _propack]$ wasm-nm _zpropack.cpython-310.so | grep bb
i bbcom_

Pyodide:

[w@fedora _propack]$ wasm-nm _zpropack.cpython-310-wasm32-emscripten.so | grep bb
i bbcom_
[w@fedora _propack]$ wasm-nm _cpropack.cpython-310-wasm32-emscripten.so | grep bb

So somehow, only one of the two files seems to refer to bbcom. We should check what's different!

[wolfv]

I removed the other unecessary patch that tried to import pyodide js.

I tried around a bit, and the same js apply() error happens with the following code:

# Fail out of bounds?
from scipy.sparse import lil_matrix
from scipy.sparse.linalg import spsolve
from numpy.linalg import solve, norm
from numpy.random import rand

A = lil_matrix((1000, 1000))
A[0, :100] = rand(100)
A[1, 100:200] = A[0, :100]
A.setdiag(rand(1000))


A = A.tocsr()
b = rand(1000)
# fail with js apply error?
x = spsolve(A, b)

And after the first apply() issues, it says something about memory acces out of bounds.

[wolfv]

I manually tested the tests:

from scipy.linalg import lapack

print(lapack.dlamch("Epsilon-Machine"))

Gives the same

ERROR CATCHED TypeError: Cannot read properties of undefined (reading 'apply')
    at stubs.<computed> (pyjs_main.js:9:40190)
    at f2pywrapdlamch_ (007a6452:0xf2675)
    at f2py_rout__flapack_dlamch (007a6452:0xd3d1c)
    at fortran_call (0003af1a:0x32c4)
    at method_call_trampoline (pyjs_main.js:9:31331)
    at pyjs_main.wasm:0x261c11
    at pyjs_main.wasm:0x32b557
    at pyjs_main.wasm:0x328da5
    at pyjs_main.wasm:0x322f53
    at pyjs_main.wasm:0x322266

as the optimize functions. Maybe something interacting with Fortran that we need to solve in general.

[wolfv]

OK, I looked a bit further, and it seems that none of the symbols of clapack_all.so are loaded and that's why the call to dlamch_ fails.

[DerThorsten]

Uhhm ist the lib in the packed js? If not, we might be on a too old empack Version.

[wolfv]

@DerThorsten I think that is not the problem. I think the problem is that clapack_all.so is not properly linked in the right places.

[wolfv]

Btw. we should improve emscripten to throw better error messages for this case

[wolfv]

I think this is the minimal reproducer:

from scipy.linalg import _flapack
_flapack.dlamch('E')

[wolfv]

things are working fine if I manually "preload" the symbols from clapack_all.so in the global space:

  var deps = await waitRunDependency()

  Module.loadDynamicLibrary("/home/w/micromamba/envs/emx/lib/clapack_all.so",{
                fs: themodule.FS,
                global: true,
                nodelete: true
            });

would love to understand how to do that automatically / properly.

[wolfv]

I have high hopes that the tests are passing now! :)

[wolfv]

locally, when running the tests in the pyjs REPL, they are passign for me. No idea what's going on here