forked from dattaz/libzim_wasm
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix makefile, add workflow to build ASM and WASM versions, update dep…
…endencies (#14)
- Loading branch information
Showing
22 changed files
with
6,293 additions
and
112 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,25 +1,71 @@ | ||
| name: Docker Image CI | ||
| # Workflow to build and publish the libzim ASM and WASM arttefacts together with the JavaScript wrapper. | ||
| # If this workflow is triggered by the creation of a draft release, then the artefacts are uploaded to the release assets. | ||
| # If it is triggered by a push or pull request to master, or manually, then the artefacts are archived under the corresponding Action. | ||
|
|
||
| name: Build and publish release artefacts (Docker) | ||
|
|
||
| on: | ||
| # push: | ||
| # branches: [ "master" ] | ||
| # pull_request: | ||
| # branches: [ "master" ] | ||
| push: | ||
| branches: [ master ] | ||
| tags: | ||
| - 'v*' # Tag push events matching v*, i.e. v1.0, v20.15.10 | ||
| pull_request: | ||
| branches: [ master ] | ||
| workflow_dispatch: | ||
| inputs: | ||
| version: | ||
| description: | | ||
| If you wish to create a draft release, set the tag version, like v9.9.9 (must not be an exisitng tag). | ||
| If left blank or incorrect format, archives will be archived instead of being uploaded to Releases. | ||
| required: false | ||
| default: '' | ||
|
|
||
| # Define top-level environment vars we can refer to below | ||
| env: | ||
| VERSION: ${{ github.ref_name }} | ||
| DISPATCH_VERSION: ${{ github.event.inputs.version }} | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| jobs: | ||
|
|
||
| build: | ||
|
|
||
| name: Build and publish W/ASM artefacts | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| # Customizes the Emscripten docker container via the Dockerfile in this repo | ||
| - name: Build the Docker image | ||
| run: docker build -t "docker-emscripten-libzim:v3" ./docker | ||
| - name: Compile the libzim WASM | ||
| run: docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) -it docker-emscripten-libzim:v3 make | ||
| - name: Publish the WASM to draft release | ||
| # Creates the ASM and WASM artefacts, and the JS wrappers, using the Makefile in this repo | ||
| - name: Compile the libzim WASM artefacts | ||
| run: docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) docker-emscripten-libzim:v3 make | ||
| - name: List directories with updated archives | ||
| run: | | ||
| echo -e "\nList ./ :" | ||
| ls -l | ||
| echo -e "\nList ./tests/prototype/ :" | ||
| ls -l tests/prototype/ | ||
| echo -e "\nList ./tests/test_large_file_access/ :" | ||
| ls -l tests/test_large_file_access/ | ||
| # If we are not creating a release, archive the artefacts under this Action run | ||
| - name: Archive build artefacts | ||
| if: | | ||
| github.event_name == 'pull_request' || github.event_name == 'push' && ! startsWith(github.ref_name, 'v') | ||
| || ! startsWith(github.event.inputs.version, 'v') | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: libzim-wasm-artefacts | ||
| path: | | ||
| libzim-wasm.* | ||
| libzim-asm.* | ||
| tests/test_large_file_access/large_file_access.* | ||
| # Otherwise, zip the artefacts into respective packages (asm and wasm), create and upload releases | ||
| - name: Zip the artefacts and create draft release | ||
| id: zip-release | ||
| if: github.event_name == 'push' && startsWith(github.ref_name, 'v') || startsWith(github.event.inputs.version, 'v') | ||
| run: | | ||
| echo 'If the run was successful, we will build a publish action here, to publish:' | ||
| ls -l a.out.* | ||
| if [[ ! $VERSION =~ ^v?[0-9.]+ ]]; then | ||
| VERSION=$DISPATCH_VERSION | ||
| fi | ||
| # Create a draft release and upload zipped artefacts as release assets | ||
| chmod +x ./scripts/create_draft_release.sh | ||
| ./scripts/create_draft_release.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,26 +1,35 @@ | ||
| # Prototype of libzim in Webassembly | ||
| # Prototype of libzim in WebAssembly (WASM) | ||
|
|
||
| Demo at https://openzim.github.io/javascript-libzim/ | ||
| This Repository provides the source code and utilities for compiling the [ZIM File](https://wiki.openzim.org/wiki/ZIM_file_format) reader [lbizim](https://wiki.openzim.org/wiki/Libzim) from C++ to [WebAssembly](https://developer.mozilla.org/en-US/docs/WebAssembly) (and [ASM.js](https://developer.mozilla.org/en-US/docs/Games/Tools/asm.js)). | ||
|
|
||
| It uses WORKERFS as FS with emscripten and run in a web worker, file object is mount before run, and name is passed as argument. | ||
| A prototype in HTML/JS, for testing the WASM version, is provided at https://openzim.github.io/javascript-libzim/tests/prototyep/. This prototype uses WORKERFS as the Emscripten File System and runs in a Web Worker. The file object is mounted before run, and the name is passed as argument. | ||
|
|
||
| There is also an HTML/JS utility for testing the ability of Emscripten File Systems to read large files (muliti-gigabyte) at https://openzim.github.io/javascript-libzim/tests/test_large_file_access/. | ||
|
|
||
| ## Steps to recompile manually | ||
|
|
||
| - Install emscripten : https://kripken.github.io/emscripten-site/docs/getting_started/downloads.html | ||
| - Install dependencies necessary for compilation. On ubuntu 18.04, you need to activate universe repository and : | ||
| - Install dependencies necessary for compilation. On ubuntu 18.04, you need to activate universe repository and: | ||
|
|
||
| ``` | ||
| sudo apt install ninja-build meson pkg-config python3 autopoint libtool autoconf | ||
| sudo apt install zlib1g-dev libicu-dev libxapian-dev liblzma-dev | ||
| ``` | ||
|
|
||
| - activate emscripten environment variables with something like `source ./emsdk_env.sh` | ||
| - run `make` | ||
|
|
||
| ## Steps to recompile with Docker | ||
| While being at the root of this repository : | ||
| - Build the Docker image with the provided Dockerfile (based on https://hub.docker.com/r/emscripten/emsdk , which is based on Debian) : | ||
|
|
||
| While being at the root of this repository: | ||
| - Build the Docker image with the provided Dockerfile (based on https://hub.docker.com/r/emscripten/emsdk, which is based on Debian): | ||
|
|
||
| ``` | ||
| docker build -t "docker-emscripten-libzim:v3" ./docker | ||
| ``` | ||
| - Run the build with : | ||
|
|
||
| - Run the build with: | ||
|
|
||
| ``` | ||
| docker run --rm -v $(pwd):/src -v /tmp/emscripten_cache/:/home/emscripten/.emscripten_cache -u $(id -u):$(id -g) -it docker-emscripten-libzim:v3 make | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
Oops, something went wrong.