add dockerfiles and hosted docker images for python/julia/r/scala (ap…

…ache#5574)

docker/.gitignore

@@ -0,0 +1,2 @@
+Dockerfile.*
+!Dockerfile.in.*

docker/Dockerfiles/Dockerfile.in.julia

@@ -0,0 +1,7 @@
+# -*- mode: dockerfile -*-
+# part of the dockerfile to install the julia binding
+
+COPY install/julia.sh install/
+RUN install/julia.sh
+ENV MXNET_HOME /mxnet
+RUN julia -e 'Pkg.add("MXNet")'

docker/Dockerfiles/Dockerfile.in.lib.cpu

@@ -0,0 +1,10 @@
+# -*- mode: dockerfile -*-
+# dockerfile to build libmxnet.so on CPU
+FROM ubuntu:14.04
+
+COPY install/cpp.sh install/
+RUN install/cpp.sh
+
+RUN git clone --recursive https://github.com/dmlc/mxnet && cd mxnet && \
+    make -j$(nproc) && \
+    rm -r build

docker/Dockerfiles/Dockerfile.in.lib.gpu

@@ -0,0 +1,9 @@
+# -*- mode: dockerfile -*-
+# dockerfile to build libmxnet.so on GPU
+FROM nvidia/cuda:8.0-cudnn5-devel
+
+COPY install/cpp.sh install/
+RUN install/cpp.sh
+
+RUN git clone --recursive https://github.com/dmlc/mxnet && cd mxnet && \
+    make -j$(nproc) USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1

docker/Dockerfiles/Dockerfile.in.python

@@ -0,0 +1,6 @@
+# -*- mode: dockerfile -*-
+# part of the dockerfile to install the python binding
+
+COPY install/python.sh install/
+RUN install/python.sh
+ENV PYTHONPATH=/mxnet/python

docker/Dockerfiles/Dockerfile.in.r-lang

@@ -0,0 +1,7 @@
+# -*- mode: dockerfile -*-
+# part of the dockerfile to install the r binding
+
+COPY install/r.sh install/
+ADD https://raw.githubusercontent.com/dmlc/mxnet/master/R-package/DESCRIPTION  install/
+RUN install/r.sh
+RUN cd mxnet && make rpkg && R CMD INSTALL mxnet_current_r.tar.gz

docker/Dockerfiles/Dockerfile.in.scala

@@ -0,0 +1,7 @@
+# -*- mode: dockerfile -*-
+# part of the dockerfile to install the scala binding
+
+COPY install/scala.sh install/
+RUN install/scala.sh
+
+RUN cd mxnet && make scalapkg

docker/README.md

@@ -1,58 +1,113 @@
 # Docker images for MXNET
 
-Pre-built docker images are available at https://hub.docker.com/r/dmlc/mxnet/
-
 ## How to use
 
-1. First pull the pre-built image
+First make sure [docker](https://docs.docker.com/engine/installation/) is
+installed. The docker plugin
+[nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is required to run on
+Nvidia GPUs.
+
+Pre-built docker containers are available at https://hub.docker.com/r/mxnet/
+
+For example, the following command launches a container with the Python package
+installed. It will pull the docker images from docker hub if it does not exist
+locally.
+
+```bash
+docker run -ti --rm mxnet/python
+```
+
+Then you can run MXNet in python, e.g.:
+
+```bash
+# python -c 'import mxnet as mx; a = mx.nd.ones((2,3)); print((a*2).asnumpy())'
+[[ 2.  2.  2.]
+ [ 2.  2.  2.]]
+```
+
+If the host machine has at least one GPU installed and `nvidia-docker` is installed, namely
+`nvidia-docker run --rm nvidia/cuda nvidia-smi` runs successfully, then you can
+run a container with GPU supports
+
+```bash
+nvidia-docker run -ti --rm mxnet/python:gpu
+```
+
+Now you can run the above example in `GPU 0`:
+
+```bash
+# python -c 'import mxnet as mx; a = mx.nd.ones((2,3), mx.gpu(0)); print((a*2).asnumpy())'
+[[ 2.  2.  2.]
+ [ 2.  2.  2.]]
+```
+
+## Hosted containers
+
+All images are based on Ubuntu 14.04. The `gpu` tag is built with CUDA 8.0 and
+cuDNN 5.
+
+### Python
 
-   ```bash
-   docker pull dmlc/mxnet
-   ```
-2. Then we can run the python shell in the docker
+Hosted at https://hub.docker.com/r/mxnet/python/
 
-   ```bash
-   docker run -ti dmlc/mxnet python
-   ```
-   For example
-   ```bash
-   $ docker run -ti dmlc/mxnet python
-   Python 2.7.6 (default, Jun 22 2015, 17:58:13)
-   [GCC 4.8.2] on linux2
-   Type "help", "copyright", "credits" or "license" for more information.
-   >>> import mxnet as mx
-   import mxnet as mx
-   >>> quit()
-   quit()
-   ```
+Python versions: 2.7.12 and 3.5.2.
 
-   Note: One may get the error message `libdc1394 error: Failed to initialize
-   libdc1394`, which is due to opencv and can be ignored.
+Available tags:
 
-3. Train a model on MNIST to check everything works
+- mxnet/python
+- mxnet/python:gpu
 
-   ```
-   docker run dmlc/mxnet python /mxnet/example/image-classification/train_mnist.py
-   ```
+### R
 
-If the host machine has Nvidia GPUs, we can use `dmlc/mxnet:cuda`, which has both CUDA and CUDNN installed.
-To launch the docker, we need to install [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) first.
+Hosted at https://hub.docker.com/r/mxnet/r-lang/
 
-1. Pull the image
+R version: 3.3.3
 
-   ```bash
-   docker pull dmlc/mxnet:cuda
-   ```
+Available tags:
 
-2. Train MNIST on GPU 0
+- mxnet/r-lang
+- mxnet/r-lang:gpu
 
-   ```bash
-   nvidia-docker run dmlc/mxnet:cuda python /mxnet/example/image-classification/train_mnist.py --gpus 0
-   ```
+
+### Julia
+
+Hosted at https://hub.docker.com/r/mxnet/julia/
+
+Julia version: 0.5.1
+
+Available tags:
+
+- mxnet/julia
+- mxnet/julia:gpu
+
+#### Scala
+
+Hosted at https://hub.docker.com/r/mxnet/scala/
+
+Scala version: 2.11.8
+
+Available tags:
+
+- mxnet/scala
 
 ## How to build
 
+The following command build the default Python package
+
+```bash
+./tool.sh build python cpu
+```
+
+Run `./tool.sh` for more details. Use
+
+
+Tips: The following commands stop all docker containers and delete all docker images.
+
+```bash
+docker stop $(docker ps -a -q)
+docker rm $(docker ps -a -q)
+```
+
 ```bash
-docker build -t dmlc/mxnet:cpu cpu
-docker build -t dmlc/mxnet:cuda cuda
+docker rmi $(docker images -a -q)
 ```

docker/install/cpp.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# libraries for building mxnet c++ core on ubuntu
+
+apt-get update && apt-get install -y \
+    build-essential git libatlas-base-dev libopencv-dev \
+    libcurl4-openssl-dev libgtest-dev cmake wget unzip
+
+cd /usr/src/gtest && cmake CMakeLists.txt && make && cp *.a /usr/lib

docker/install/julia.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# install libraries for mxnet's julia package on ubuntu
+
+# the julia version shipped with ubuntu (version 0.4) is too low. so download a
+# new version
+# apt-get install -y julia
+
+wget -q https://julialang.s3.amazonaws.com/bin/linux/x64/0.5/julia-0.5.1-linux-x86_64.tar.gz
+tar -zxf julia-0.5.1-linux-x86_64.tar.gz
+rm julia-0.5.1-linux-x86_64.tar.gz
+ln -s $(pwd)/julia-6445c82d00/bin/julia /usr/bin/julia

docker/install/python.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# install libraries for mxnet's python package on ubuntu
+
+apt-get update && apt-get install -y python-dev python3-dev
+
+# the version of the pip shipped with ubuntu may be too lower, install a recent version here
+cd /tmp && wget https://bootstrap.pypa.io/get-pip.py && python3 get-pip.py && python2 get-pip.py
+
+pip2 install nose pylint numpy nose-timer requests
+pip3 install nose pylint numpy nose-timer requests

docker/install/r.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+# install libraries for mxnet's r package on ubuntu
+
+echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list
+gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
+gpg -a --export E084DAB9 | apt-key add -
+
+apt-get update
+apt-get install -y r-base r-base-dev libxml2-dev libxt-dev libssl-dev
+
+cd "$(dirname "${BASH_SOURCE[0]}")"
+
+if [ ! -f "./DESCRIPTION" ]; then
+    cp ../../R-package/DESCRIPTION .
+fi
+
+Rscript -e "install.packages('devtools', repo = 'https://cran.rstudio.com')"
+Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cran.rstudio.com')); install_deps(dependencies = TRUE)"

docker/install/scala.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# install libraries for mxnet's scala package on ubuntu
+
+apt-get install -y maven default-jdk
+
+wget http://downloads.lightbend.com/scala/2.11.8/scala-2.11.8.deb
+dpkg -i scala-2.11.8.deb
+rm scala-2.11.8.deb

docker/run.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# Build and push all docker containers
+
+DEVICES=('cpu' 'gpu')
+LANGUAGES=('python' 'julia' 'r-lang' 'scala')
+for DEV in "${DEVICES[@]}"; do
+    for LANG in "${LANGUAGES[@]}"; do
+        ./tool.sh build ${LANG} ${DEV}
+        ./tool.sh push ${LANG} ${DEV}
+    done
+done