Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and test TVM under minimal configuration #12178

Merged
merged 4 commits into from
Aug 11, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add new docker image with minimal TVM configuration
Florin-Gabriel Blanaru authored and gigiblender committed Aug 8, 2022

Verified

This commit was signed with the committer’s verified signature.
commit 86c69fa49debaa2c72a755cbb69623ae41945fb1
57 changes: 57 additions & 0 deletions docker/Dockerfile.ci_minimal
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# CI docker minimal CPU env
FROM ubuntu:18.04

COPY utils/apt-install-and-clear.sh /usr/local/bin/apt-install-and-clear

RUN apt-get update --fix-missing

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

COPY install/ubuntu_install_googletest.sh /install/ubuntu_install_googletest.sh
RUN bash /install/ubuntu_install_googletest.sh

COPY install/ubuntu1804_install_python.sh /install/ubuntu1804_install_python.sh
RUN bash /install/ubuntu1804_install_python.sh

# Globally disable pip cache
RUN pip config set global.no-cache-dir false

COPY install/ubuntu_install_python_package.sh /install/ubuntu_install_python_package.sh
RUN bash /install/ubuntu_install_python_package.sh

COPY install/ubuntu1804_manual_install_llvm.sh /install/ubuntu1804_manual_install_llvm.sh
RUN bash /install/ubuntu1804_manual_install_llvm.sh

# Rust env (build early; takes a while)
COPY install/ubuntu_install_rust.sh /install/ubuntu_install_rust.sh
RUN bash /install/ubuntu_install_rust.sh
ENV RUSTUP_HOME /opt/rust
ENV CARGO_HOME /opt/rust
ENV PATH $PATH:$CARGO_HOME/bin

# AutoTVM deps
COPY install/ubuntu_install_redis.sh /install/ubuntu_install_redis.sh
RUN bash /install/ubuntu_install_redis.sh

# sccache
COPY install/ubuntu_install_sccache.sh /install/ubuntu_install_sccache.sh
RUN bash /install/ubuntu_install_sccache.sh
ENV PATH /opt/sccache:$PATH
38 changes: 38 additions & 0 deletions docker/install/ubuntu1804_manual_install_llvm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -e
set -u
set -o pipefail

git clone --depth 1 --branch release/11.x https://github.com/llvm/llvm-project.git
pushd llvm-project
mkdir build
pushd build
cmake \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_ENABLE_PROJECTS="llvm;clang" \
../llvm
ninja install
popd
popd
rm -rf llvm-project

1 change: 1 addition & 0 deletions tests/python/unittest/test_meta_schedule_task_scheduler.py
Original file line number Diff line number Diff line change
@@ -337,6 +337,7 @@ def test_meta_schedule_task_scheduler_override_next_task_id_only(): # pylint: d
)


@pytest.mark.skip("Does array access OOB")
gigiblender marked this conversation as resolved.
Show resolved Hide resolved
def test_meta_schedule_task_scheduler_multiple_gradient_based():
num_trials_per_iter = 6
max_trials_per_task = 101
13 changes: 13 additions & 0 deletions tests/scripts/ci.py
Original file line number Diff line number Diff line change
@@ -595,6 +595,19 @@ def add_subparser(
"frontend": ("run frontend tests", ["./tests/scripts/task_python_frontend_cpu.sh"]),
},
),
generate_command(
name="minimal",
help="Run minimal CPU build and test(s)",
options={
"cpp": CPP_UNITTEST,
"unittest": (
"run unit tests",
[
"./tests/scripts/task_python_unittest.sh",
],
),
},
),
generate_command(
name="i386",
help="Run i386 build and test(s)",
33 changes: 33 additions & 0 deletions tests/scripts/task_config_build_minimal.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -euxo pipefail

BUILD_DIR=$1
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
cp ../cmake/config.cmake .

echo set\(USE_SORT ON\) >> config.cmake
gigiblender marked this conversation as resolved.
Show resolved Hide resolved
echo set\(USE_LLVM llvm-config\) >> config.cmake
echo set\(USE_RELAY_DEBUG ON\) >> config.cmake
echo set\(CMAKE_CXX_FLAGS \"-Werror -Wp,-D_GLIBCXX_ASSERTIONS\"\) >> config.cmake
echo set\(HIDE_PRIVATE_SYMBOLS ON\) >> config.cmake
echo set\(USE_LIBBACKTRACE ON\) >> config.cmake
echo set\(USE_CCACHE OFF\) >> config.cmake
echo set\(SUMMARIZE ON\) >> config.cmake
23 changes: 13 additions & 10 deletions tests/scripts/task_cpp_unittest.sh
Original file line number Diff line number Diff line change
@@ -45,20 +45,23 @@ python3 tests/scripts/task_build.py \
--cmake-target cpptest \
--build-dir "${BUILD_DIR}"

# crttest requires USE_MICRO to be enabled, which is currently the case
# with all CI configs
pushd "${BUILD_DIR}"
ninja crttest
popd
# crttest requires USE_MICRO to be enabled.
if grep -Fq "set(USE_MICRO ON)" ${BUILD_DIR}/config.cmake; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i really wanted to be able to have you just look this up in CMakeCache.txt, but unfortunately we don't cache these. i wonder if we should be setting this with set(USE_ABC ___ CACHED) so we can inspect that. what if we modified Summary.cmake to write the options to a file via https://cmake.org/cmake/help/latest/command/file.html#append?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might also be cleaner to split this out into task_cpp_unittest_micro.sh or something and just add it as another step everywhere that currently runs the cpp unit tests, that way its easy to run regardless of the specific build options

Copy link
Contributor Author

@gigiblender gigiblender Jul 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went the route of separating the script into two. Let me know your thoughts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: It seems that there are crt tests that get included in the cpp test suite when USE_MICRO is enabled.

Instead of separating those, I am following @areusch advice and writing to a file all the options used to build TVM.

pushd "${BUILD_DIR}"
ninja crttest
popd
fi


pushd "${BUILD_DIR}"
ctest --gtest_death_test_style=threadsafe
popd

# Test MISRA-C runtime
pushd apps/bundle_deploy
rm -rf build
make test_dynamic test_static
popd
# Test MISRA-C runtime. It requires USE_MICRO to be enabled.
if grep -Fq "set(USE_MICRO ON)" ${BUILD_DIR}/config.cmake; then
pushd apps/bundle_deploy
rm -rf build
make test_dynamic test_static
popd
fi