adding openmc wasm executable #1
Comments
|
Looks like libraries need to be built and linked |
|
Perhaps add hdf5 with this repo |
|
I found the hdf5 library binaries in this path and exported the path in the Dockerfile but still I face the missing library issues:
|
|
After that, I used this GitHub repo https://github.com/usnistgov/libhdf5-wasm.git to install the hdf5 library but still faced a missing library issue. Commands that I run in the Dockerfile. This hdf5 library needs a cmake version greater or equal to 3.24. |
|
Thanks for your efforts, I've tried adding some lines to the openmc CMakeLists.txt file so that it fetches the libhdf5-wasm as suggested in the https://github.com/usnistgov/libhdf5-wasm readme file I added these lines and saved a new CMakelists.txt locally but this also appeared to not work as I still got the same missing libraries on the error message The dockerfile has a few more lines in it and now looks like this |
|
not sure but perhaps the cmakelists file needs another few changes to |
|
I resolve these three missing library issues:
using the below commands: |
|
many thanks @bilal-07 are you able to provide the full dockerfile for that 🙏 |
|
Here is the complete Dockerfile. @shimwell |
|
Thanks for the dockerfile, I built it and see that it has less missing libraries 🎉 now many thanks |
|
I've tried to add zlib (called lz in error message) by downloading the release and building it with emake make. The libz.a file can be found with the find command just like libhdf5.a and libhdf5_hl.a Then I've tried adding a DZLIB_INCLUDE_DIR command on the cmake command but it doesn't appear to help with the error messages which are still So I don't think cmake is finding the .a files Here is the latest dockerfile |
|
I've been trying here is the dockerfile |
|
updated the docker image to emsdk 3.1.49 but there error persist # docker build -t openmc_wasm
FROM emscripten/emsdk:3.1.49
# Set the working directory
WORKDIR /app
# Install dependencies
RUN apt-get update -y && \
apt-get install -y build-essential g++ cmake libhdf5-dev libpng-dev mpich libmpich-dev libhdf5-serial-dev libhdf5-mpich-dev
# Download and build HDF5
RUN wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.3/src/hdf5-1.12.3.tar.gz && \
tar -xzvf hdf5-1.12.3.tar.gz && \
cd hdf5-1.12.3 && \
./configure --prefix=/emsdk/upstream/emscripten/system/include/hdf5 && \
make -j && \
make install
# Clone your repository
WORKDIR /app
RUN git clone --recurse-submodules https://github.com/openmc-dev/openmc.git
# Create build directory
WORKDIR /app/openmc/build
RUN embuilder build zlib
RUN embuilder build libpng
RUN find / -name libhdf5.a
RUN find / -name libhdf5_hl.a
RUN find / -name libz.a
# Configure the build with emcmake
RUN emcmake cmake -DHDF5_ROOT=/emsdk/upstream/emscripten/system/include/hdf5/bin ..
RUN make -j VERBOSE=1 |
|
removed x86 packages and downloading a prebuilt hdf5 wasm error is now docker file is now |
|
The libz.a and libsz.a libraries used in building libhdf5 are included in the download, next to libhdf5.a and libhdf5_hl.a in /lib mkdir hdf5
tar -xzf HDF5-1.14.2-Emscripten.tar.gz -C hdf5
export HDF5_ROOT=$(readlink -f hdf5)
mkdir build
cd build
emcmake cmake \
-DOPENMC_BUILD_TESTS=OFF \
-DCMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES=$HDF5_ROOT/include \
-DHDF5_LIBRARIES="$HDF5_ROOT/lib/libhdf5.a;$HDF5_ROOT/lib/libz.a;$HDF5_ROOT/lib/libsz.a" \
-DHDF5_HL_LIBRARIES="$HDF5_ROOT/lib/libhdf5_hl.a" \
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=TRUE \
-DHDF5_DIR="$HDF5_ROOT/cmake" \
-DZLIB_LIBRARY="$HDF5_ROOT/lib/libz.a" \
-DZLIB_INCLUDE_DIR="$HDF5_ROOT/include" \
-DCMAKE_CXX_LINK_FLAGS="-sALLOW_MEMORY_GROWTH=1" \
..
emmake make -j Note that I disabled tests because I got a duplicate symbol error in test_interpolation.cpp:
|
|
Also, the other tests work - if I comment out test_interpolate in tests/cpp_unit_tests/CMakeLists.txt, the other tests build and execute and pass: > cd bin
> node test_distribution.js
Randomness seeded to: 1928082616
===============================================================================
All tests passed (14 assertions in 2 test cases)
> node test_tally.js
Randomness seeded to: 3656487369
===============================================================================
All tests passed (12 assertions in 1 test case)
> node test_file_utils.js
Randomness seeded to: 3256973951
===============================================================================
All tests passed (12 assertions in 3 test cases) |
|
One more note - in your README that you mention that h5py is not available in Jupyterlite - this is not true for recent versions (h5py has been added to the base pyodide distribution, as well as newer Jupyterlite) |
|
There is one more flag that is needed in order to make the executable openmc.js work - you have to enable memory growth. I have edited my comment above to include For testing it with node, I also added <?xml version="1.0"?>
<materials>
<cross_sections>/home/bbm/nndc_hdf5/cross_sections.xml</cross_sections>
...
</materials>After that, I can run the wasm executable with node: > node openmc.js --geometry-debug ../../tests/regression_tests/density
%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%
############### %%%%%%%%%%%%%%%%%%%%%%%%
################## %%%%%%%%%%%%%%%%%%%%%%%
################### %%%%%%%%%%%%%%%%%%%%%%%
#################### %%%%%%%%%%%%%%%%%%%%%%
##################### %%%%%%%%%%%%%%%%%%%%%
###################### %%%%%%%%%%%%%%%%%%%%
####################### %%%%%%%%%%%%%%%%%%
####################### %%%%%%%%%%%%%%%%%
###################### %%%%%%%%%%%%%%%%%
#################### %%%%%%%%%%%%%%%%%
################# %%%%%%%%%%%%%%%%%
############### %%%%%%%%%%%%%%%%
############ %%%%%%%%%%%%%%%
######## %%%%%%%%%%%%%%
%%%%%%%%%%%
| The OpenMC Monte Carlo Code
Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
License | https://docs.openmc.org/en/latest/license.html
Version | 0.14.1-dev
Git SHA1 | 9830efaf2aadf68ffde9a054388bacef20ab5645
Date/Time | 2023-11-27 12:30:30
Reading settings XML file...
Reading cross sections XML file...
Reading materials XML file...
Reading geometry XML file...
Reading U235 from /home/bbm/nndc_hdf5/U235.h5
Reading U238 from /home/bbm/nndc_hdf5/U238.h5
Reading H1 from /home/bbm/nndc_hdf5/H1.h5
Minimum neutron data temperature: 294 K
Maximum neutron data temperature: 294 K
Preparing distributed cell instances...
Reading plot XML file...
Writing summary.h5 file...
WARNING: Cell overlap checking is ON.
Maximum neutron transport energy: 20000000 eV for U235
Initializing source particles...
====================> K EIGENVALUE SIMULATION <====================
Bat./Gen. k Average k
========= ======== ====================
1/1 1.20883
2/1 1.10631
3/1 1.12003
4/1 1.12568
5/1 1.07936
6/1 1.14136
7/1 1.10553 1.12344 +/- 0.01791
8/1 1.11293 1.11994 +/- 0.01092
9/1 1.07564 1.10887 +/- 0.01350
10/1 1.05217 1.09753 +/- 0.01543
Creating state point statepoint.10.h5...
=======================> TIMING STATISTICS <=======================
Total time for initialization = 2.5344e-01 seconds
Reading cross sections = 1.9321e-01 seconds
Total time in simulation = 7.3894e-02 seconds
Time in transport only = 6.7888e-02 seconds
Time in inactive batches = 3.9527e-02 seconds
Time in active batches = 3.4367e-02 seconds
Time synchronizing fission bank = 1.2590e-03 seconds
Sampling source sites = 1.1632e-03 seconds
SEND/RECV source sites = 6.8888e-05 seconds
Time accumulating tallies = 2.9593e-05 seconds
Time writing statepoints = 3.7460e-03 seconds
Total time for finalization = 1.1320e-06 seconds
Total time elapsed = 3.3428e-01 seconds
Calculation Rate (inactive) = 126496 particles/second
Calculation Rate (active) = 145487 particles/second
============================> RESULTS <============================
k-effective (Collision) = 1.09897 +/- 0.01112
k-effective (Track-length) = 1.09753 +/- 0.01543
k-effective (Absorption) = 1.10874 +/- 0.00713
Combined k-effective = 1.11006 +/- 0.01303
Leakage Fraction = 0.52180 +/- 0.00399
==================> CELL OVERLAP CHECK SUMMARY <===================
Cell ID No. Overlap Checks
1 9461
2 17234
3 16477
4 10603
There were 0 cells with less than 10 overlap checksNote that this binary is now portable to any platform! |
|
Wow thank you so much for looking into this. This is really going to change everything for us over in opemc world from our tutorial hosting to our porting to windows. I would have never figured out these tricky aspects of the code especially those extra flags. The only thing that confuses me a little is the h5py as I was trying to add it to over here to get it working with xeus-python + JupyterLite as I thought that was the only way. I'm going to look into how h5py was added to pyodide distribution, as well as newer Jupyterlite and see if that helps get h5py working on working for xeus-python. I'm going to tinker with everything, many many thanks |
|
I helped with getting h5py added to pyodide, so let me know if there's anything I can do to help with your problem, as well. |
|
I'm seeing a couple of errors when running the latest docker file first error is related to PNG I'm not sure why as cmake reports libpng is found at Next error is during the Docker file and modified cmakelists.txt attached for reproduction # docker build -t openmc_wasm .
# docker run -it --entrypoint /bin/bash openmc_wasm
FROM emscripten/emsdk:3.1.49
WORKDIR /app
# Install dependencies
RUN apt-get update -y && \
apt-get install -y build-essential g++ cmake
RUN embuilder build libpng
RUN git clone --depth 1 --recurse-submodules https://github.com/openmc-dev/openmc.git
RUN wget https://github.com/usnistgov/libhdf5-wasm/releases/download/v0.4.2_3.1.43/HDF5-1.14.2-Emscripten.tar.gz && \
tar -f HDF5-1.14.2-Emscripten.tar.gz -z -C /emsdk/upstream/emscripten/cache/sysroot -x
RUN export HDF5_ROOT=/emsdk/upstream/emscripten/cache/sysroot/hdf5
WORKDIR /app/openmc
# this file has lines 513 to 516 commented out so PNG is not looked for
# COPY CMakeLists.txt .
# Create build directory
WORKDIR /app/openmc/build
# Configure the build with emcmake
RUN emcmake cmake \
-DOPENMC_BUILD_TESTS=OFF \
-DCMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES=$HDF5_ROOT/include \
-DHDF5_LIBRARIES="$HDF5_ROOT/lib/libhdf5.a;$HDF5_ROOT/lib/libz.a;$HDF5_ROOT/lib/libsz.a" \
-DHDF5_HL_LIBRARIES="$HDF5_ROOT/lib/libhdf5_hl.a" \
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=TRUE \
-DHDF5_DIR="$HDF5_ROOT/cmake" \
-DZLIB_LIBRARY="$HDF5_ROOT/lib/libz.a" \
-DZLIB_INCLUDE_DIR="$HDF5_ROOT/include" \
-DCMAKE_CXX_LINK_FLAGS="-sALLOW_MEMORY_GROWTH=1;-sNODERAWFS=1;-sFORCE_FILESYSTEM=1" \
..
RUN emmake make -j VERBOSE=1 |
|
Sorry - I was being clumsy stomping on the existing CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES... which was causing the PNG::PNG error I think (it was populating incorrects include dirs). It turns out it's not even needed. Also, running "export HDF5_ROOT" in there was not properly setting the variable in the Dockerfile. Here is a modified Dockerfile that successfully builds openmc.js (including png support?) # docker build -t openmc_wasm .
# docker run -it --entrypoint /bin/bash openmc_wasm
FROM emscripten/emsdk:3.1.49
WORKDIR /app
# Install dependencies
RUN apt-get update -y && \
apt-get install -y build-essential g++ cmake
RUN embuilder build libpng
RUN git clone --depth 1 --recurse-submodules https://github.com/openmc-dev/openmc.git
RUN wget https://github.com/usnistgov/libhdf5-wasm/releases/download/v0.4.2_3.1.43/HDF5-1.14.2-Emscripten.tar.gz && \
tar -f HDF5-1.14.2-Emscripten.tar.gz -z -C /emsdk/upstream/emscripten/cache/sysroot -x
ARG HDF5_ROOT=/emsdk/upstream/emscripten/cache/sysroot
WORKDIR /app/openmc
# this file has lines 513 to 516 commented out so PNG is not looked for
# COPY CMakeLists.txt .
# Create build directory
WORKDIR /app/openmc/build
# Configure the build with emcmake
RUN emcmake cmake \
-DOPENMC_BUILD_TESTS=OFF \
-DHDF5_LIBRARIES="$HDF5_ROOT/lib/libhdf5.a;$HDF5_ROOT/lib/libz.a;$HDF5_ROOT/lib/libsz.a" \
-DHDF5_HL_LIBRARIES="$HDF5_ROOT/lib/libhdf5_hl.a" \
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=TRUE \
-DHDF5_DIR="$HDF5_ROOT/cmake" \
-DZLIB_LIBRARY="$HDF5_ROOT/lib/libz.a" \
-DZLIB_INCLUDE_DIR="$HDF5_ROOT/include" \
-DCMAKE_EXE_LINKER_FLAGS="-sALLOW_MEMORY_GROWTH=1 -sNODERAWFS=1 -sFORCE_FILESYSTEM=1" \
..
RUN emmake make -j VERBOSE=1 |
|
The testing issue has been fixed over on the openmc repo so this docker should now work with ... confirming the docker file now builds with tests on |
|
latest docker file |
|
Next step is to run this openmc.js file through in this browser https://shimwell.github.io/openmc-online-demo/lab/index.html |
|
Did you want to run it from a command-line in a terminal in the jupyterlite session? |
|
I've copied the files locally to my host (non docker) with these commands I have also uploaded them to the repo here in the ideal world I would be able to run this openmc executable with the standard openmc python command However I think that will need changes on the openmc side. So running it from a command-line in a terminal in the jupyterlite session I see you used |
|
Unfortunately But I notice one can run javascript in jupyterlite with some magic command |
|
If you want to use the python interface, I think it will be important to build libopenmc.so as a shared library so that python can call it with You also have to manually link in all the libraries, as CMake and. Emscripten do not currently have a built-in way to build shared libraries. Here is a Dockerfile that builds # docker build -t openmc_wasm .
# docker run -it --entrypoint /bin/bash openmc_wasm
FROM emscripten/emsdk:3.1.49
WORKDIR /app
# Install dependencies
RUN apt-get update -y && \
apt-get install -y build-essential g++ cmake
RUN embuilder --pic build libpng
RUN git clone --depth 2 --recurse-submodules https://github.com/openmc-dev/openmc.git
RUN wget https://github.com/usnistgov/libhdf5-wasm/releases/download/v0.4.3_3.1.43/HDF5-1.14.2-Emscripten.tar.gz && \
tar -f HDF5-1.14.2-Emscripten.tar.gz -z -C /emsdk/upstream/emscripten/cache/sysroot -x
ARG HDF5_ROOT=/emsdk/upstream/emscripten/cache/sysroot
WORKDIR /app/openmc
# Create build directory
WORKDIR /app/openmc/build
# Configure the build with emcmake
RUN emcmake cmake \
-DOPENMC_BUILD_TESTS=OFF \
-DHDF5_LIBRARIES="$HDF5_ROOT/lib/libhdf5.a;$HDF5_ROOT/lib/libz.a;$HDF5_ROOT/lib/libsz.a" \
-DHDF5_HL_LIBRARIES="$HDF5_ROOT/lib/libhdf5_hl.a" \
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=TRUE \
-DHDF5_DIR="$HDF5_ROOT/cmake" \
-DZLIB_LIBRARY="$HDF5_ROOT/lib/libz.a" \
-DZLIB_INCLUDE_DIR="$HDF5_ROOT/include" \
-DCMAKE_CXX_STANDARD=14 \
..
RUN emmake make -j VERBOSE=1 libopenmc
RUN emcc -sSIDE_MODULE=1 -sWASM_BIGINT \
lib/libopenmc.a lib/libfmt.a lib/libpugixml.a \
$HDF5_ROOT/lib/libhdf5.a \
$HDF5_ROOT/lib/libz.a \
$HDF5_ROOT/lib/libsz.a \
$HDF5_ROOT/lib/wasm32-emscripten/pic/libpng.a \
-o libopenmc.soNotes
|
|
After you build the .so as above, you might want to embed it in the openmc WASM wheel file - for pyodide, this would involve
|
|
It might be tricky to get the python openmc code to be able to call the openmc executable from a javascript/webassembly environment. As far as I know os.system or subprocess don't work at all. If you want to execute your code, it will probably have to be done by importing the main function from the openmc executable into the javascript environment where you're also running pyodid/xeus python. Then you can call main through the python->javascript api, e.g. "import js; js.openmc._main()". This is very different from how the executable is now called from python! |
|
Many thanks for all the pointers, I got the new dockerfile and built the libopenmc.so but that is as far as I got for now. Will keep going tomorrow |
|
openmc-0.14.1.dev0-cp311-cp311-emscripten_3_1_45_wasm32.zip This wheel should work for pyodide |
ideally we would have the openmc executable compiled to web assembly
I tried with this docker container
build and run with these commands
from within the docker command promt follow instructions here
https://gist.github.com/WesThorburn/00c47b267a0e8c8431e06b14997778e4
this errors with the following message
The text was updated successfully, but these errors were encountered: