Dockerized implementation

.gitignore

@@ -1,3 +1,6 @@
 .vscode
 *.pyc
-*__pycache__*
+*__pycache__*
+*.egg-info
+.devcontainer/
+.env

Dockerfile

@@ -0,0 +1,39 @@
+FROM osrf/ros:kinetic-desktop-full
+
+LABEL author="Olivier Lamarre"
+LABEL description="Run ENAV dataset utilities"
+
+# Dependencies
+RUN apt-get update \
+    && apt-get install -y \
+    cython \
+    git \
+    libeigen3-dev \
+    libgdal-dev \
+    python-pip \
+    python-tk \
+    ros-kinetic-grid-map \
+    ros-kinetic-interactive-marker-twist-server \
+    ros-kinetic-moveit-ros-visualization \
+    ros-kinetic-robot-localization \
+    tmux \
+    && rm -rf /var/lib/apt/lists/*
+
+# Catkin workspace & custom package
+RUN mkdir -p /root/catkin_ws/src
+COPY ./enav_ros /root/catkin_ws/src/enav_ros
+
+RUN git clone https://github.com/MHarbi/bagedit.git /root/catkin_ws/src/bagedit
+RUN cd /root/catkin_ws/src/bagedit && git checkout f97b21050826f65757040750cecc165abd67c0d2
+
+RUN . /opt/ros/kinetic/setup.sh && cd /root/catkin_ws && catkin_make
+
+RUN echo "source /opt/ros/kinetic/setup.bash" >> /root/.bashrc
+RUN echo "source /root/catkin_ws/devel/setup.bash" >> /root/.bashrc
+
+# Python data fetching script
+COPY ./enav_utilities /root/enav_utilities
+RUN cd /root/enav_utilities && pip install -e .
+
+# Disable default entrypoint from parent ROS image
+ENTRYPOINT []

README.md

@@ -20,61 +20,71 @@ This dataset is described in details in our paper [*The Canadian Planetary Emula
 
 ## Overview
 
-The data can be downloaded from the dataset's official [web page](http://www.starslab.ca/enav-planetary-dataset/), which contains the data both in human-readable and rosbag format. This repository provides tools to interact with the rosbag files.
+The dataset is further described in the project's official [web page](http://www.starslab.ca/enav-planetary-dataset/) and can be downloaded from a dedicated [IEEE DataPort page](https://ieee-dataport.org/open-access/canadian-planetary-emulation-terrain-energy-aware-rover-navigation-dataset) (requires creating a free account). The dataset is available in both rosbag and human-readable formats. This repository provides tools to interact with the rosbag files; we let users interested in the human-readable-formatted data develop their own utilities.
 
-Data fetching and plotting alone in generic Python formats can be accomplished using our custom fetching script (which wraps the rosbag module). Advanced data visualization and interaction is enabled using our lightweight ROS package (tested with ROS Kinetic on Ubuntu 16.04):
+We provide a Docker container in which rosbags can be played and visualized using our custom ROS package, or parsed using our python utility scripts:
 
 ![enav_ros](https://media.giphy.com/media/YlBFpSSD9qxbCCCpyp/giphy.gif)
 
-Lastly, this dataset also includes four different aerial maps of the test environment at a resolution of 0.2 meters per pixel: color, elevation, slope magnitude and slope orientation maps. Every map is georeferenced and is available in a `.tif` format. Tools to load them in Python or import them in ROS as a single [grid_map](https://github.com/ANYbotics/grid_map) message are also included:
+Lastly, this dataset also includes four different aerial maps of the test environment at a resolution of 0.2 meters per pixel: color, elevation, slope magnitude and slope orientation maps. Every map is georeferenced and is available in a `.tif` format. Tools to broadcast them as ROS topics (as [grid_map](https://github.com/ANYbotics/grid_map) messages) or simply load them using Python are also included:
 
 ![enav_maps](https://media.giphy.com/media/j4w8J6OvbReBvQyqn0/giphy.gif)
 
+## Setup
 
-## Pre-requisites
+As the dataset was originally collected with ROS Kinetic with a Ubuntu 16.04 machine, we provide a Docker container to play rosbags and/or extract specific data streams with Python.
 
-1. ROS installation ([ROS Kinetic](http://wiki.ros.org/kinetic/Installation/Ubuntu) tested);
-2. Python 3 (version 3.7 tested). We recommend using a Python environment manager like [pyenv](https://github.com/pyenv/pyenv);
+1. Store downloaded rosbags in a single directory and the georeferenced maps in a dedicated subdirectory.
 
-## Installation
+```sh
+tree /path/to/dataset/dir
 
-1. Clone this repository (including the submodules) in the `src` directory of your catkin workspace:
+├── run1_base_new.bag
+├── run2_base_new.bag
+├── run3_base_new.bag
+├── run4_base_new.bag
+├── ...
+├── maps
+    ├── aspect_utm_20cm.tif
+    ├── dem_utm_20cm.tif
+    ├── mosaic_utm_20cm.tif
+    └── slope_utm_20cm.tif
+```
+
+> Note: obviously, you don't need to download all the rosbags - just the ones you need.
 
-   ```sh
-   cd ~/catkin_ws/src
-   git clone --recurse-submodules https://github.com/utiasSTARS/enav-planetary-dataset.git
-   ```
+2. Clone the current repository and store the location of the downloaded dataset in a `.env` file in the project's root directory:
 
-2. In a Python 3 environment, the required Python 3 modules can be installed using pip and our [requirements file](#):  
-   `pip install -r requirements.txt`  
+```sh
+git clone https://github.com/utiasSTARS/enav-planetary-dataset.git
+cd enav-planetary-dataset
+
+echo "ENAV_DATASET_DIR='/path/to/enav_dataset/dir'" > .env
+```
 
-3. Install the required dependencies ([husky_msgs](http://wiki.ros.org/husky_msgs), [robot_localization](http://wiki.ros.org/robot_localization and [grid_map](https://github.com/ANYbotics/grid_map)) ROS packages):
+3. If not already done, [install Docker](https://docs.docker.com/engine/install/). Then, set up X server permissions:
 
-   ```sh
-   sudo apt-get update
-   sudo apt-get install ros-kinetic-husky-msgs ros-kinetic-robot-localization ros-kinetic-grid-map
-   ```
+```sh
+xhost +local:root
+```
 
-4. Build the enav_ros package:
+Run a container and enter it:
 
-   ```sh
-   cd ~/catkin_ws
+```sh
+docker compose run --rm enav
+```
 
-   # If you are using catkin build:
-   catkin build enav_ros
-   source ~/catkin_ws/setup.bash
+Note that the dataset directory is now mounted at `/enav_dataset` in the container.
 
-   # Alternatively, using catkin_make:
-   catkin_make --pkg enav_ros
-   source ~/catkin_ws/setup.bash
-   ```
+Rosbag interactions are done from inside the container:
 
-## Documentation
+- Playing rosbags and launching ROS visualization interfaces is documented in the `enav_ros` package's [README file](enav_ros/README.md).
 
-Documentation on how to fetch & plot the data collected by the rover or how to show our aerial maps (along with sample Python scripts) can be found in the [enav_utilities](enav_utilities) subdirectory. Similarly, instructions related to data visualization using our lightweight ROS package can be found in the [enav_ros](enav_ros) subdirectory.
+- Extracting data from rosbags with Python is documented in the `enav_utilities` package's [README file](enav_utilities/README.md).
 
 ## Citation
-```
+
+```txt
 @article{lamarre2020canadian,
    author = {Lamarre, Olivier and Limoyo, Oliver and Mari{\'c}, Filip and Kelly, Jonathan},
    title = {{The Canadian Planetary Emulation Terrain Energy-Aware Rover Navigation Dataset}},

dev/Dockerfile.dev

@@ -0,0 +1,44 @@
+# Docker file dor development purposes with the ENAV project.
+# Mounts the repository in a running container, as opposed to copying its
+# contents (which is what the deployed version does)
+
+FROM osrf/ros:kinetic-desktop-full
+
+LABEL author="Olivier Lamarre"
+LABEL description="Development for ENAV dataset utilities"
+
+# Dependencies
+RUN apt-get update \
+    && apt-get install -y \
+    build-essential \
+    curl \
+    cython \
+    git \
+    libeigen3-dev \
+    libgdal-dev \
+    python-pip \
+    python-tk \
+    ros-kinetic-grid-map \
+    ros-kinetic-interactive-marker-twist-server \
+    ros-kinetic-moveit-ros-visualization \
+    ros-kinetic-robot-localization \
+    tmux \
+    vim \
+    && rm -rf /var/lib/apt/lists/*
+
+# Catkin workspace & custom ROS package
+RUN mkdir -p /root/catkin_ws/src
+# COPY ./enav_ros /root/catkin_ws/src/enav_ros
+
+RUN git clone https://github.com/MHarbi/bagedit.git /root/catkin_ws/src/bagedit
+RUN cd /root/catkin_ws/src/bagedit && git checkout f97b21050826f65757040750cecc165abd67c0d2
+
+RUN . /opt/ros/kinetic/setup.sh && cd /root/catkin_ws && catkin_make
+
+RUN echo "source /opt/ros/kinetic/setup.bash" >> /root/.bashrc
+RUN echo "source /root/catkin_ws/devel/setup.bash" >> /root/.bashrc
+
+WORKDIR /workspace
+
+# Disable default entrypoint from parent ROS image
+ENTRYPOINT []

dev/docker-compose.dev.yml

@@ -0,0 +1,52 @@
+services:
+  enav:
+    build:
+      context: ..  # Project root is the build context
+      dockerfile: dev/Dockerfile.dev
+
+    # X server setup, inspired by https://wiki.ros.org/docker/Tutorials/GUI
+    environment:
+      - DISPLAY
+      - QT_X11_NO_MITSHM=1
+      - DEBUG=true
+
+    stdin_open: true  # --interactive (-i) flag
+    tty: true  # --tty (-t) flag
+
+    volumes:
+      - ..:/workspace
+      - /tmp/.X11-unix:/tmp/.X11-unix:rw  # X server
+      - ${ENAV_DATASET_DIR}:/enav_dataset
+
+    # Build & install mounted project volume (bash keeps the container running)
+    command: >
+      bash -c "
+      ln -s /workspace/enav_ros /root/catkin_ws/src &&
+      cd /root/catkin_ws && source /opt/ros/kinetic/setup.sh && catkin_make &&
+      cd /workspace/enav_utilities &&
+      pip install -e . &&
+      echo 'ENAV utilities mounted & installed, container up & running' &&
+      bash"
+
+
+    # ports:
+    #   - "8000:8000"
+    # entrypoint: ["bash"]
+
+    # image: olamarre/enav:latest
+    # container_name: enav
+
+
+    # # X server setup, inspired by https://wiki.ros.org/docker/Tutorials/GUI
+    # environment:
+    #   - DISPLAY
+    #   - QT_X11_NO_MITSHM=1
+
+    # volumes:
+    #   - /tmp/.X11-unix:/tmp/.X11-unix:rw  # X server
+    #   - ${ENAV_DATASET_DIR}:/enav_dataset
+    #   - ./:/dev/enav-planetary-dataset
+
+    # # ports:
+    # #   - 8080:8080
+    # command: bash

dev/readme.md

@@ -0,0 +1,77 @@
+# dev workflow
+
+From repo's root directory:
+
+1. Create .env file with path to dataset:
+
+```sh
+echo "ENAV_DATASET_DIR='/path/to/dataset/dir'" > .env
+```
+
+2. Create dev image & start container (-d runs it in the background):
+
+```sh
+docker compose -f dev/docker-compose.dev.yml --env-file .env up -d
+```
+
+3. Open container in vscode: Bottom left: `Open a Remove Window` > `Attach to a running container`
+
+Useful container commands:
+
+```sh
+# Check running container name
+docker container ls
+
+# Open the running container in a separate shell
+docker container exec -it [name] bash
+
+# Stop a running container
+docker container stop [name]
+
+# Remove a container
+docker container rm [name]
+```
+
+Useful image commands:
+
+```sh
+# Check image name
+docker images
+
+# Remove image
+docker image rm [name]
+```
+
+## Re-build & push new main image
+
+1. Remove older images, if any:
+
+```sh
+docker image rm enav
+```
+
+2. Regenerate a new image:
+
+```sh
+docker build -t enav --no-cache .
+```
+
+3. Update image name & tag to point to olamarre's dockerhub repo:
+
+```sh
+docker tag enav:latest olamarre/enav:latest
+```
+
+4. Push changes to dockerhub
+
+```sh
+docker push olamarre/enav
+```
+
+5. Erase local images & test installation with docker-compose file:
+
+```sh
+docker image rm enav olamarre/enav
+
+docker compose run --rm enav
+```

docker-compose.yml

@@ -0,0 +1,18 @@
+services:
+  enav:
+    image: olamarre/enav:latest
+    container_name: enav
+
+    stdin_open: true  # --interactive (-i) flag
+    tty: true  # --tty (-t) flag
+
+    # X server setup, inspired by https://wiki.ros.org/docker/Tutorials/GUI
+    environment:
+      - DISPLAY
+      - QT_X11_NO_MITSHM=1
+
+    volumes:
+      - /tmp/.X11-unix:/tmp/.X11-unix:rw  # X server
+      - ${ENAV_DATASET_DIR}:/enav_dataset
+
+    command: bash  # keep the container session open