Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to documentation and docker setups from my getting spun up #101

Closed
wants to merge 12 commits into from
6 changes: 4 additions & 2 deletions .github/workflows/integration_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,7 @@ jobs:
sudo rm -rf /opt/ghc
- name: Check out code
uses: actions/checkout@v2
- name: Compile and run tests
run: docker build -t mbari_lrauv . -f docker/tests/Dockerfile
- name: Compile tests
run: docker build -t osrf_lrauv . -f docker/tests/Dockerfile
- name: Run tests
run: docker run --rm osrf_lrauv
41 changes: 39 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,55 @@ Optionally, you may choose to build this repository using Docker, for
convenience.
Make sure you have a recent version of [Docker](https://docs.docker.com/) and
[nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker)
installed. Next to get started simply run the following command.
installed.
And [rocker](https://github.com/osrf/rocker) installed.

```
docker/build_and_run_docker.sh
```

To join in a separate terminal, remember to source Ignition and the workspace:
```
docker/join.sh mbari_lrauv
docker/join.sh osrf_lrauv
. /home/ign_ws/install/setup.bash
. /home/colcon_ws/install/setup.bash
```


### Empty World from Source

~ 90 minutes from scratch

~ 70 seconds with only workspace changes

```
cd docker
docker build -t osrf_lrauv -f empty_world/Dockerfile ..
rocker --nvidia --x11 --user osrf_lrauv bash
ign launch lrauv_world.ign
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The launch file was removed in #102 , we should use plain ign gazebo commands now

```

### Unit tests dockerfile

Enter test environment

```
docker build -t osrf_lrauv_tests -f tests/Dockerfile ..
rocker --nvidia --x11 --user osrf_lrauv_tests bash
```

Then run anything you want in the environment, unit tests are in `./build/lrauv_ignition_plugins`
```
./build/lrauv_ignition_plugins/test_buoyancy_action
```

Or the main entry point

```
ign launch lrauv_world.ign
```


## To build

To run the code in this repository natively without Docker, make sure you have
Expand Down
50 changes: 3 additions & 47 deletions docker/build_and_run_docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,55 +23,11 @@

# Make sure processes in the container can connect to the x server
# Necessary so gazebo can create a context for OpenGL rendering (even headless)
XAUTH=/tmp/.docker.xauth
if [ ! -f $XAUTH ]
then
xauth_list=$(xauth nlist $DISPLAY)
xauth_list=$(sed -e 's/^..../ffff/' <<< "$xauth_list")
if [ ! -z "$xauth_list" ]
then
echo "$xauth_list" | xauth -f $XAUTH nmerge -
else
touch $XAUTH
fi
chmod a+r $XAUTH
fi

DOCKER_OPTS=

# Get the current version of docker
DOCKER_VER=$(docker version --format '{{.Server.Version}}')
if dpkg --compare-versions 19.03 gt "$DOCKER_VER"
then
echo "Docker version is less than 19.03, using nvidia-docker2 runtime"
if ! dpkg --list | grep nvidia-docker2
then
echo "Please either update docker-ce to a version greater than 19.03 or install nvidia-docker2"
exit 1
fi
DOCKER_OPTS="$DOCKER_OPTS --runtime=nvidia"
else
DOCKER_OPTS="$DOCKER_OPTS --gpus all"
fi

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

cd $DIR
# Build the docker image
docker build -t mbari_lrauv -f $DIR/empty_world/Dockerfile ..
docker build -t osrf_lrauv -f $DIR/tests/Dockerfile ..

rocker --nvidia --x11 --user --user-override-name=developer --user-preserve-home -- osrf_lrauv

# Run the image
docker run -it \
-e DISPLAY \
-e QT_X11_NO_MITSHM=1 \
-e XAUTHORITY=$XAUTH \
-v "$XAUTH:$XAUTH" \
-v "/tmp/.X11-unix:/tmp/.X11-unix" \
-v "/etc/localtime:/etc/localtime:ro" \
-v "/dev/input:/dev/input" \
--network host \
--rm \
--privileged \
--security-opt seccomp=unconfined \
$DOCKER_OPTS \
mbari_lrauv
54 changes: 4 additions & 50 deletions docker/debug_integration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,60 +23,14 @@

# Make sure processes in the container can connect to the x server
# Necessary so gazebo can create a context for OpenGL rendering (even headless)
XAUTH=/tmp/.docker.xauth
if [ ! -f $XAUTH ]
then
xauth_list=$(xauth nlist $DISPLAY)
xauth_list=$(sed -e 's/^..../ffff/' <<< "$xauth_list")
if [ ! -z "$xauth_list" ]
then
echo "$xauth_list" | xauth -f $XAUTH nmerge -
else
touch $XAUTH
fi
chmod a+r $XAUTH
fi

DOCKER_OPTS=

# Get the current version of docker
DOCKER_VER=$(docker version --format '{{.Server.Version}}')
if dpkg --compare-versions 19.03 gt "$DOCKER_VER"
then
echo "Docker version is less than 19.03, using nvidia-docker2 runtime"
if ! dpkg --list | grep nvidia-docker2
then
echo "Please either update docker-ce to a version greater than 19.03 or install nvidia-docker2"
exit 1
fi
DOCKER_OPTS="$DOCKER_OPTS --runtime=nvidia"
else
DOCKER_OPTS="$DOCKER_OPTS --gpus all"
fi
# exit if the docker image doesn't build
set -e

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

cd $DIR

# exit if the docker image doesn't build
set -e

# Build the docker image
docker build -t mbari_lrauv_debug -f $DIR/debug_integration/Dockerfile ..
docker build -t osrf_lrauv_tests -f $DIR/tests/Dockerfile ..

# Run the image
docker run -it \
-e DISPLAY \
-e QT_X11_NO_MITSHM=1 \
-e XAUTHORITY=$XAUTH \
-v "$XAUTH:$XAUTH" \
-v "/tmp/.X11-unix:/tmp/.X11-unix" \
-v "/etc/localtime:/etc/localtime:ro" \
-v "/dev/input:/dev/input" \
-v "$DIR/../../results:/results"\
--network host \
--rm \
--privileged \
--security-opt seccomp=unconfined \
$DOCKER_OPTS \
mbari_lrauv_debug
rocker --nvidia --x11 --user --user-override-name=developer --user-preserve-home -- osrf_lrauv_tests tmuxinator start debug_and_plot -n debug_session -p src/lrauv/docker/tests/debug_integration_mux.yml
79 changes: 0 additions & 79 deletions docker/debug_integration/Dockerfile

This file was deleted.

12 changes: 6 additions & 6 deletions docker/empty_world/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,12 @@ COPY . /home/colcon_ws/src
WORKDIR /home/colcon_ws
RUN [ "/bin/bash" , "-c" , \
"source ${IGN_WS}/install/setup.bash \
&& colcon build \
&& colcon build --cmake-args='-DBUILD_TESTING=ON' \
&& apt-get -qq clean" ]

SHELL ["/bin/bash", "-c"]
COPY docker/entrypoint.bash /

# start ignition
ENTRYPOINT [ "/bin/bash" , "-c" , \
"source /home/colcon_ws/install/setup.bash \
&& ign gazebo buoyant_tethys.sdf" ]
# setup ignition environment
ENTRYPOINT ["/entrypoint.bash"]

CMD ign gazebo buoyant_tethys.sdf
16 changes: 16 additions & 0 deletions docker/entrypoint.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
set -e

# Empty world dockerfile
if [ -z /home/colcon_ws/install/setup.bash ]; then
source /home/colcon_ws/install/setup.bash
# tests dockerfile
else #if [ -z ~/lrauv_ws/install/setup.bash ]; then
source /home/developer/lrauv_ws/install/setup.bash
# Make sure we're in the right workingdir
cd /home/developer/lrauv_ws
fi

exec "$@"


28 changes: 15 additions & 13 deletions docker/tests/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,15 @@ RUN apt-get update \
build-essential \
curl \
gnupg2 \
libpcl-dev \
lsb-release \
python3-numpy \
tmux \
tmuxinator \
tzdata \
wget \
python3-numpy
&& rm -rf /var/lib/apt/lists/* \
&& apt-get -qq clean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth it cleaning it here, considering we'll do an update below again?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this removes the contents of the apt cache from the layer, making the docker container much slimmer. Otherwise you just end up with outdated caches in every layer of the container.


# Add Ignition's latest packages, which may be more up-to-date than the ones from the MBARI image
RUN /bin/sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-stable.list' && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to add the nightly repo now, for Garden

Expand All @@ -47,16 +52,10 @@ RUN /bin/sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable

# Install and upgrade the latest Ignition binaries
RUN apt-get -qq update && apt-get -q -y install \
ignition-garden && apt-get dist-upgrade -y

# Install PCL
RUN apt-get update \
&& apt-get install -y \
libpcl-dev
ignition-garden\
&& rm -rf /var/lib/apt/lists/* \
&& apt-get -qq clean && apt-get dist-upgrade -y

# Clean up apt
RUN rm -rf /var/lib/apt/lists/* \
&& apt-get -qq clean

USER $USERNAME

Expand All @@ -69,9 +68,12 @@ RUN rm -rf build log install src/lrauv
# Use current version of the code
COPY . src/lrauv

# Build image
# Build workspace
RUN [ "/bin/bash" , "-c" , "colcon build --cmake-args='-DBUILD_TESTING=true'" ]

COPY docker/entrypoint.bash /
# setup ignition environment
ENTRYPOINT ["/entrypoint.bash"]

# Run tests
RUN [ "/bin/bash" , "-c" , ". ~/lrauv_ws/install/setup.sh; colcon test --event-handlers console_direct+" ]
RUN [ "/bin/bash" , "-c" , ". ~/lrauv_ws/install/setup.sh; colcon test-result" ]
CMD [ "/bin/bash" , "-c" , "colcon test --event-handlers console_direct+; colcon test-result" ]
chapulina marked this conversation as resolved.
Show resolved Hide resolved