Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow compiling with Rust code #3137

Merged
merged 61 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
4ebc2d6
Squashed Rust impl
nyurik Jul 24, 2024
e5b038b
cmake
nyurik Aug 3, 2024
cad4ff4
cxx
nyurik Aug 4, 2024
dc0defe
wip
nyurik Aug 4, 2024
46ac304
add armv7-linux-androideabi
nyurik Aug 5, 2024
0aacaaf
add rust to qt docker build
nyurik Aug 5, 2024
2847d27
fix
nyurik Aug 5, 2024
4752a7d
fix
nyurik Aug 5, 2024
cc8eb72
build all qt
nyurik Aug 5, 2024
e07f5c8
build all qt
nyurik Aug 5, 2024
0afbe40
wip
nyurik Aug 6, 2024
c157da8
wip
nyurik Aug 7, 2024
61f7d3f
wip
nyurik Aug 7, 2024
5e404f4
wip
nyurik Aug 7, 2024
f712840
Merge branch 'main' into rust-css-color
louwers Jan 10, 2025
f2b7b7b
Add Bazel support
louwers Jan 10, 2025
d3836b6
Add back license
louwers Jan 10, 2025
90ba3ef
Cleanup unused code
louwers Jan 10, 2025
6f618ec
Revert some files
louwers Jan 10, 2025
729dafb
CI fixes
louwers Jan 10, 2025
c7aceda
Remove README.md
louwers Jan 10, 2025
5fc141e
revert qt.cmake
louwers Jan 10, 2025
671a41f
revert README.md
louwers Jan 10, 2025
8ed4303
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2025
53a33c8
revert
louwers Jan 10, 2025
a5853d9
revert
louwers Jan 10, 2025
bda511d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2025
a2fb48c
PR comments
louwers Jan 10, 2025
a43b2e7
Merge branch 'rust-css-color' of github.com:nyurik/maplibre-native in…
louwers Jan 10, 2025
627c4a1
Merge branch 'rust-css-color' of github.com:louwers/maplibre-native i…
louwers Jan 10, 2025
fca3163
fix merge oops
louwers Jan 10, 2025
643de09
fix setup-java
louwers Jan 10, 2025
29a2bb2
PR comments, fix CMake
louwers Jan 10, 2025
bfafa2b
Add rustfmt to pre-commit hooks
louwers Jan 10, 2025
3e06aa3
change CMake file
louwers Jan 10, 2025
6d848bb
put docs under right heading
louwers Jan 10, 2025
f74e5bb
fix .pre-commit.yaml
louwers Jan 10, 2025
a6162cb
Update docker/Dockerfile
louwers Jan 13, 2025
2ebf02a
Update rustutils/cpp/generate.sh
louwers Jan 13, 2025
4d64771
Update src/mbgl/util/color.rs.cpp
louwers Jan 13, 2025
0101522
allow modules, remove generates sources, update license
louwers Jan 13, 2025
cc5bf2a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 13, 2025
5ae0610
remove rust from android-ci workflow
louwers Jan 14, 2025
96757f3
add extra_target_triples to configure Rust toolchain iOS
louwers Jan 14, 2025
11dafe6
add basic CI for Rust
louwers Jan 14, 2025
32b377f
Merge branch 'rust-css-color' of github.com:louwers/maplibre-native i…
louwers Jan 14, 2025
8d016f3
yaml oopsie
louwers Jan 14, 2025
7355d6b
formatting
louwers Jan 14, 2025
1f2aeb9
fix linux-ci
louwers Jan 14, 2025
32f3f1c
improve rust.md
louwers Jan 14, 2025
06b4bef
improve rustutils.cmake
louwers Jan 14, 2025
4228b6e
improve linux-ci
louwers Jan 14, 2025
2f608a6
use cargo fmt
louwers Jan 14, 2025
991f1e1
add newlines
louwers Jan 14, 2025
af5b67b
Fix issue in linux-ci
louwers Jan 14, 2025
6afef70
fix typo in linux-ci
louwers Jan 14, 2025
038fa16
Merge branch 'main' of github.com:maplibre/maplibre-native into rust-…
louwers Jan 15, 2025
014f2ba
Fix color.rs.cpp impl, add test
louwers Jan 15, 2025
04091e0
fix factor premultiplication
louwers Jan 16, 2025
e671d23
Add Rust toolchains for iOS, build cxxbridge with Bazel
louwers Jan 16, 2025
22df4de
fix rustutils/BUILD.bazel
louwers Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7.3.1
8.0.0
15 changes: 13 additions & 2 deletions .github/workflows/linux-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ jobs:
- name: Get all Linux files that have changed
if: github.event_name != 'workflow_dispatch'
id: changed-files

uses: tj-actions/changed-files@v45
with:
files_yaml_from_source_file: .github/changed-files.yml
Expand All @@ -52,7 +53,7 @@ jobs:
strategy:
fail-fast: true
matrix:
renderer: [legacy, drawable, vulkan]
renderer: [legacy, drawable, vulkan, drawable-rust]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -91,6 +92,11 @@ jobs:
- if: matrix.renderer == 'drawable'
run: echo renderer_flag_cmake=-DMLN_DRAWABLE_RENDERER=ON >> "$GITHUB_ENV"

- if: matrix.renderer == 'drawable-rust'
run: |
echo "renderer_flag_cmake=-DMLN_DRAWABLE_RENDERER=ON -DMLN_USE_RUST=ON" >> "$GITHUB_ENV"
cargo install cxxbridge-cmd

- if: matrix.renderer == 'legacy'
run: echo renderer_flag_cmake=-DMLN_LEGACY_RENDERER=ON >> "$GITHUB_ENV"

Expand Down Expand Up @@ -158,7 +164,12 @@ jobs:

- name: Run render test
id: render_test
run: xvfb-run -a build/mbgl-render-test-runner --manifestPath=metrics/linux-${{ matrix.renderer }}.json
run: |
renderer="${{ matrix.renderer }}"
if [[ "$renderer" == *-rust ]]; then
renderer=${renderer%-rust}
fi
xvfb-run -a build/mbgl-render-test-runner --manifestPath=metrics/linux-"$renderer".json

- name: Upload render test result
if: always() && steps.render_test.outcome == 'failure'
Expand Down
7 changes: 7 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ repos:
hooks:
- id: swiftformat
args: [--swiftversion, "5.8"]
- repo: local
hooks:
- id: rustfmt
name: rustfmt
entry: bash -c 'cd rustutils && cargo fmt' --
language: rust
types: [rust]
ci:
# sometimes fails https://github.com/keith/pre-commit-buildifier/issues/13
skip: [buildifier]
Expand Down
23 changes: 21 additions & 2 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
load("@aspect_rules_js//js:defs.bzl", "js_binary", "js_library", "js_run_binary")
load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "string_flag")
load("@npm//:defs.bzl", "npm_link_all_packages")
load(
"//bazel:core.bzl",
Expand Down Expand Up @@ -124,7 +124,6 @@ cc_library(
":mbgl-core-generated-private-artifacts",
":mbgl-core-generated-public-artifacts",
"//vendor:boost",
"//vendor:csscolorparser",
"//vendor:earcut.hpp",
"//vendor:eternal",
"//vendor:mapbox-base",
Expand All @@ -149,6 +148,13 @@ cc_library(
"//vendor:metal-cpp",
],
"//conditions:default": [],
}) + select({
":rust": [
"//rustutils:rustutilslib",
],
"//conditions:default": [
"//vendor:csscolorparser",
],
}),
)

Expand Down Expand Up @@ -195,6 +201,19 @@ config_setting(
},
)

bool_flag(
name = "use_rust",
build_setting_default = False,
visibility = ["//visibility:public"],
)

config_setting(
name = "rust",
flag_values = {
"//:use_rust": "true",
},
)

exports_files(
[
"LICENSE.md",
Expand Down
23 changes: 16 additions & 7 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ option(MLN_LEGACY_RENDERER "Include the legacy rendering pathway" ON)
option(MLN_DRAWABLE_RENDERER "Include the drawable rendering pathway" OFF)
option(MLN_USE_UNORDERED_DENSE "Use ankerl dense containers for performance" ON)
option(MLN_USE_TRACY "Enable Tracy instrumentation" OFF)
option(MLN_USE_RUST "Use components in Rust" OFF)

if (MLN_WITH_CLANG_TIDY)
find_program(CLANG_TIDY_COMMAND NAMES clang-tidy)
Expand Down Expand Up @@ -940,7 +941,7 @@ list(APPEND SRC_FILES
${PROJECT_SOURCE_DIR}/src/mbgl/util/bounding_volumes.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/chrono.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/client_options.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/color.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/color$<IF:$<BOOL:${MLN_USE_RUST}>,.rs.cpp,.cpp>
${PROJECT_SOURCE_DIR}/src/mbgl/util/constants.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/convert.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/event.cpp
Expand Down Expand Up @@ -1458,6 +1459,10 @@ include(${PROJECT_SOURCE_DIR}/vendor/vector-tile.cmake)
include(${PROJECT_SOURCE_DIR}/vendor/wagyu.cmake)
include(${PROJECT_SOURCE_DIR}/vendor/metal-cpp.cmake)

if(MLN_USE_RUST)
include(${PROJECT_SOURCE_DIR}/rustutils/rustutils.cmake)
endif()

target_link_libraries(
mbgl-core
PRIVATE
Expand All @@ -1468,7 +1473,6 @@ target_link_libraries(
Mapbox::Base::cheap-ruler-cpp
mbgl-compiler-options
mbgl-vendor-boost
mbgl-vendor-csscolorparser
mbgl-vendor-earcut.hpp
mbgl-vendor-eternal
mbgl-vendor-parsedate
Expand All @@ -1479,6 +1483,7 @@ target_link_libraries(
mbgl-vendor-vector-tile
mbgl-vendor-wagyu
$<$<BOOL:${MLN_WITH_METAL}>:mbgl-vendor-metal-cpp>
$<IF:$<BOOL:${MLN_USE_RUST}>,mbgl-rustutils,mbgl-vendor-csscolorparser>
PUBLIC
Mapbox::Base
Mapbox::Base::Extras::expected-lite
Expand All @@ -1490,9 +1495,8 @@ target_link_libraries(
unordered_dense
)

export(TARGETS
set(EXPORT_TARGETS
mbgl-core

mapbox-base
mapbox-base-cheap-ruler-cpp
mapbox-base-extras-expected-lite
Expand All @@ -1508,7 +1512,6 @@ export(TARGETS
mapbox-base-variant
mbgl-compiler-options
mbgl-vendor-boost
mbgl-vendor-csscolorparser
mbgl-vendor-earcut.hpp
mbgl-vendor-eternal
mbgl-vendor-parsedate
Expand All @@ -1520,10 +1523,16 @@ export(TARGETS
mbgl-vendor-wagyu
mbgl-vendor-metal-cpp
unordered_dense

FILE MapboxCoreTargets.cmake
)

if(MLN_USE_RUST)
list(APPEND EXPORT_TARGETS mbgl-rustutils rustutils)
else()
list(APPEND EXPORT_TARGETS mbgl-vendor-csscolorparser)
endif()

export(TARGETS ${EXPORT_TARGETS} FILE MapboxCoreTargets.cmake)

if(MLN_WITH_VULKAN)
include(${PROJECT_SOURCE_DIR}/vendor/vulkan.cmake)

Expand Down
35 changes: 33 additions & 2 deletions MODULE.bazel
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module(name = "maplibre")

bazel_dep(name = "apple_support", version = "1.17.0", repo_name = "build_bazel_apple_support")
bazel_dep(name = "apple_support", version = "1.17.1", repo_name = "build_bazel_apple_support")
louwers marked this conversation as resolved.
Show resolved Hide resolved
bazel_dep(name = "bazel_skylib", version = "1.7.1")
bazel_dep(name = "platforms", version = "0.0.10")
bazel_dep(name = "rules_apple", version = "3.16.1", repo_name = "build_bazel_rules_apple")
Expand Down Expand Up @@ -59,4 +59,35 @@ darwin_config = use_repo_rule("//platform/darwin:bazel/darwin_config_repository_

darwin_config(
name = "darwin_config",
)
)

bazel_dep(name = "rules_rust", version = "0.56.0")
bazel_dep(name = "cxx.rs", version = "1.0.136")
git_override(
module_name = "cxx.rs",
commit = "d54e44698c3fa5833a861cb3ae502533b92f2f57",
remote = "https://github.com/dtolnay/cxx.git",
)

rust = use_extension("@rules_rust//rust:extensions.bzl", "rust")
rust.toolchain(
edition = "2021",
extra_target_triples = [
"aarch64-apple-ios-sim",
"x86_64-apple-ios",
"aarch64-apple-ios",
"aarch64-apple-darwin",
"x86_64-apple-darwin",
],
)
use_repo(rust, "rust_toolchains")

register_toolchains("@rust_toolchains//:all")

crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate")
crate.from_cargo(
name = "crates",
cargo_lockfile = "//rustutils:Cargo.lock",
manifests = ["//rustutils:Cargo.toml"],
)
use_repo(crate, "crates")
10 changes: 8 additions & 2 deletions bazel/core.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,6 @@ MLN_CORE_SOURCE = [
"src/mbgl/util/bounding_volumes.cpp",
"src/mbgl/util/chrono.cpp",
"src/mbgl/util/client_options.cpp",
"src/mbgl/util/color.cpp",
"src/mbgl/util/constants.cpp",
"src/mbgl/util/convert.cpp",
"src/mbgl/util/event.cpp",
Expand Down Expand Up @@ -652,7 +651,14 @@ MLN_CORE_SOURCE = [
"src/mbgl/util/version.cpp",
"src/mbgl/util/version.hpp",
"src/mbgl/util/work_request.cpp",
]
] + select({
"//:rust": [
"src/mbgl/util/color.rs.cpp",
],
"//conditions:default": [
"src/mbgl/util/color.cpp",
],
})

MLN_CORE_HEADERS = [
"include/mbgl/gfx/context.hpp",
Expand Down
21 changes: 17 additions & 4 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,23 @@ RUN apt-get update \
ccache \
ninja-build \
pkg-config \
python3 \
python3-pip \
python-is-python3 \
clang-tidy \
&& : # end of the RUN cmd - easier to keep a colon at the end of the list, than to keep the backslashes in check

# This could also be `.../releases/latest/download/bazelisk-linux-amd64` for the latest version, but for predictability better hardcode it
# Detect if current CPU is x64 or ARM64 and download the appropriate binary
RUN if [ "$(uname -m)" = "aarch64" ]; then \
RUN echo "Download and install Bazel" \
&& if [ "$(uname -m)" = "aarch64" ]; then \
curl -fsSL https://github.com/bazelbuild/bazelisk/releases/download/v1.20.0/bazelisk-linux-arm64 -o /usr/local/bin/bazel ;\
else \
curl -fsSL https://github.com/bazelbuild/bazelisk/releases/download/v1.20.0/bazelisk-linux-amd64 -o /usr/local/bin/bazel ;\
fi \
&& chmod +x /usr/local/bin/bazel \
&& :

WORKDIR /app

ARG USERNAME=user
ARG USER_UID=1000
Expand All @@ -50,11 +53,21 @@ RUN groupadd --force --gid $USER_GID $USERNAME \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME

# This allows users to `docker run` without specifying -u and -g
USER $USERNAME

RUN pip install pre-commit

ENV RUSTUP_HOME=/home/$USERNAME/.cache/.rustup \
CARGO_HOME=/home/$USERNAME/.cache/.cargo \
PATH=/home/$USERNAME/.cache/.cargo/bin:$PATH

# As the very last step, copy the startup script
USER root
COPY startup.sh /usr/local/bin/startup.sh
RUN chmod +x /usr/local/bin/startup.sh

# This allows users to `docker run` without specifying -u and -g
USER $USERNAME

WORKDIR /app
ENTRYPOINT ["/usr/local/bin/startup.sh"]
CMD ["bash"]
42 changes: 40 additions & 2 deletions docker/startup.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,51 @@
#!/bin/sh

if [ ! -d /app/.github ] || [ ! -d /home/user/.cache ]; then
if [ ! -d /app/.github ] || [ ! -d ~/.cache ]; then
echo " "
echo "ERROR: Docker container was not started properly."
echo " From the root of this repo, run the following command."
echo " You may add any command to perform in the container at the end of this command."
echo " "
echo ' docker run --rm -it -v "$PWD:/app/" -v "$PWD/docker/.cache:/home/user/.cache" maplibre-native-image'
echo ' docker run --rm -it -v "$PWD:/app/" -v "$PWD/docker/.cache:/home/'"$USERNAME"'/.cache" maplibre-native-image'
exit 1
fi

export PATH="$PATH:~/.local/bin/"


# Work in progress: install and configure Swift and pre-commit
# Detect if current CPU is x64 or ARM64 and download the appropriate binary
#RUN echo "Download and install SWIFT" \
# && if [ "$(uname -m)" = "aarch64" ]; then \
# curl -fsSL https://download.swift.org/swift-5.10.1-release/ubuntu2204-aarch64/swift-5.10.1-RELEASE/swift-5.10.1-RELEASE-ubuntu22.04-aarch64.tar.gz \
# -o /tmp/swift.tar.gz ;\
# else \
# curl -fsSL https://download.swift.org/swift-5.10.1-release/ubuntu2204/swift-5.10.1-RELEASE/swift-5.10.1-RELEASE-ubuntu22.04.tar.gz \
# -o /tmp/swift.tar.gz ;\
# fi \
# && tar -xzf /tmp/swift.tar.gz -C / --strip-components=1 \
# && rm /tmp/swift.tar.gz \
# && :
#if [ ! -f "/app/.git/hooks/pre-commit" ]; then
# echo "Configuring pre-commit git hooks by creating a .git/hooks/pre-commit file..."
# ~/.local/bin/pre-commit install
#fi



if [ ! -f "$CARGO_HOME/env" ]; then
echo "Downloading and installing Rust..."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
fi
. "$CARGO_HOME/env"



if ! command -v cxxbridge > /dev/null; then
echo "Installing cxxbridge..."
cargo install cxxbridge-cmd
fi



exec "$@"
2 changes: 2 additions & 0 deletions docs/mdbook/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,5 @@

- [Profiling applications that use MapLibre Native](./profiling/README.md)
- [Tracy profiling](./profiling/tracy-profiling.md)

- [Rust](./rust.md)
Loading
Loading