Merge pull request #4 from ihmeuw/readme

Readme

README.md

@@ -0,0 +1,67 @@
+# cartodb-docker
+### containerization resources for CartoDB (aka Carto)
+
+This repository provides Docker images for each of the services that comprise [Carto](https://carto.com), so that the application can be deployed in a containerized environment. The setup of each service is based on [the development documentation](https://cartodb.readthedocs.io) from Carto. We recommend you consult that document for up-to-date details on how to use the application.
+
+There are subdirectories corresponding to each of the Carto services:
+- `editor`: the Carto web application (https://github.com/CartoDB/cartodb)
+- `mapsapi`: map tile API server (https://github.com/CartoDB/Windshaft-cartodb)
+- `sqlapi`: API server allowing arbitrary SQL queries against the database (https://github.com/CartoDB/CartoDB-SQL-API)
+- `postgis`: PostGIS database server with Carto extensions (https://github.com/CartoDB/cartodb-postgresql)
+- `redis`: data store used by other Carto components; an off-the-rack Redis image configured to persist data on disk
+
+In addition, we supply a simple nginx reverse-proxy server image (in the `router` subdirectory), configured to allow access to all Carto endpoints via a single base URL, and a sample `docker-compose.yml` as an example of how these services may be linked together to run the whole application.
+
+See the `README.md` file in each subdirectory for details on using the various Docker images.
+
+## Images
+
+The images are available on DockerHub:
+- [ihme/cartodb-editor](https://hub.docker.com/r/ihme/cartodb-editor/)
+- [ihme/cartodb-mapsapi](https://hub.docker.com/r/ihme/cartodb-mapsapi/)
+- [ihme/cartodb-sqlapi](https://hub.docker.com/r/ihme/cartodb-sqlapi/)
+- [ihme/cartodb-postgis](https://hub.docker.com/r/ihme/cartodb-postgis/)
+- [ihme/cartodb-redis](https://hub.docker.com/r/ihme/cartodb-redis/)
+- [ihme/cartodb-router](https://hub.docker.com/r/ihme/cartodb-router/)
+
+## Usage
+
+To run this application, you need to have a recent version of Docker installed. To spin up a bare-bones implementation locally, with default configuration, simply run this command from the project root:
+```
+docker-compose -d up
+```
+
+In a more real-world scenario, you'll want to:
+1. provide some custom configuration
+2. deploy the application in an environment where the containers can be distributed across muliple logical hosts
+
+To configure Carto to your specific needs, at a minimum you'll need to supply your own `docker-compose.yml`, or whatever configuration file(s) are needed by your container orchestration engine. That would allow options like:
+- configuring containers with custom environment variables
+- configuring containers with a custom `command` and/or `entrypoint`
+- changing the port on which the application is exposed to outside traffic
+- attaching volumes to persist data beyond the life of the containers
+- add your own containerized application(s) to consume the Carto services
+- and more
+
+## Why use this incarnation of Carto?
+
+If you're reading this, perhaps you already have an idea why you want to use Carto. If not, have a look at the [Carto homepage](https://carto.com) for a description of the platform and what it's useful for.
+
+The Carto team provides [detailed instructions](https://cartodb.readthedocs.io/en/latest/install.html) for installing the system. These instructions assume you're installing all the components on a single logical host, though, and no resources are provided for containerization. For those who want a containerized Carto, several open-source solutions do exist. Why then use this one?
+
+As of this writing, most of the alternative solutions put the entire application in a single image. That certainly makes setup simpler, but it seems contrary to the container paradigm, where each process should ideally inhabit a separate container. Packing a PostGIS database server, a Redis instance, two Node.JS API servers, and a full Ruby on Rails web application into a single image has several disadvantages:
+- the image is truly massive
+- services can't be distributed or replicated across multiple hosts, limiting performance and scalability
+- the system becomes a monolith in which all the parts are entangled
+
+This is of course not the appropriate place to detail the tradeoffs between a monolithic architecture versus a microservices approach, but the container paradigm is certainly geared towards the latter. This repo attempts to package Carto as an idiomatic containerized application, in which each service inhabits a separate container.
+
+We've also taken pains to make the images configurable and extensible. Most allow you to configure a container at runtime by modifying the `command` and/or `entrypoint`, and more advanced configuration changes can be accomplished simply by extending these images and adding one or more configuration files to your derived image.  Again, see the `README.md` in each subdirectory for details.
+
+## Versioning and image tags
+
+Each image in this repo is versioned separately. In each subdirectory you'll find a file called `VERSION` that records two fields:
+- `SOFTWARE_VERSION`: the version of the underlying piece of software we're containerizing
+- `IMAGE_VERSION`: the version of the image itself, reflecting the contents of the Dockerfile and other resources in this repo used to create the image
+
+Each image tag uses the format: `<SOFTWARE_VERSION>-<IMAGE_VERSION>`.

editor/README.md

@@ -0,0 +1,35 @@
+# cartodb-editor
+### Carto web application
+
+`ihme/cartodb-editor` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-editor/
+
+This image contains the [Carto web application](https://github.com/CartoDB/cartodb), used to administer the Carto system.
+
+## Configuration
+
+Upon starting the application, a new user will be created so that you can log into the system. For security, we recommend you change the default login credentials by passing the following environment variables to the container:
+- DEFAULT_USER (default: username)
+- PASSWORD (default: password)
+- EMAIL (default: [email protected])
+
+To set the public URL on which the application will be visible to web traffic, you can pass hostname and port to the container via these environment variables:
+- PUBLIC_HOST (default: localhost)
+- PUBLIC_PORT (default: 80)
+
+The application is configured using two YAML files, `app_config.yml` and `database.yml`. Carto supplies sample versions of these files, `app_config.yml.sample` and `database.yml.sample` in the (container) directory `/cartodb/config/`. You can configure the system to your liking by supplying your own YAML files. Simply extend this image by `COPY`ing custom config files into `/cartodb/config/`,
+and pass the names of your files via the `command` to the container like this:
+
+```dockerfile
+CMD [ \
+  "--app_config", "my_app_config.yml", \
+  "--database", "my_database.yml" \
+]
+```
+
+If desired, you can supply multiple filenames for both `--app_config` and `--database`. The configuration objects they contain will be merged. Note that with respect to merging, files earlier in the list take precedence over files later in the list.
+
+Note also that we supply some additional configuration, in the files `base.app_config.yml` and `base.database.yml`, to make this image work together with the other `cartodb-docker` images. These configuration settings have the highest precedence, overriding the same fields in any other configuration files supplied. We don't recommend you override these files, because the container startup script, `docker-entrypoint.sh`, assumes they'll be present. If you do replace them, you'll likely need to modify the startup script too or else supply your own.
+
+# Usage
+
+The container generally takes a few minutes to initialize the system after starting up. Once it has finished initializing (watch the container log for the message "Starting the application..."), you should be able to access it from a web browser via the URL `PUBLIC_HOST:PUBLIC_PORT`, filling in the values you supplied for the corresponding environment variables (NB: if `PUBLIC_PORT` is 80, you can just use the URL `PUBLIC_HOST`). At the login prompt, enter the credentials you supplied to the container as `DEFAULT_USER` and `PASSWORD`. Now you can use Carto!

mapsapi/README.md

@@ -0,0 +1,19 @@
+# cartodb-mapsapi
+### Maps API for cartodb-docker
+
+`ihme/cartodb-mapsapi` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-mapsapi/
+
+This image contains the [Carto Maps API](https://github.com/CartoDB/Windshaft-cartodb), a map tile server.
+
+## Configuration
+
+The server can be configured via a `.js` file in a specific format. Carto provides several such files as examples:
+- [development.js.example](https://github.com/CartoDB/Windshaft-cartodb/blob/master/config/environments/development.js.example)
+- [production.js.example](https://github.com/CartoDB/Windshaft-cartodb/blob/master/config/environments/production.js.example)
+- [staging.js.example](https://github.com/CartoDB/Windshaft-cartodb/blob/master/config/environments/staging.js.example)
+- [test.js.example](https://github.com/CartoDB/Windshaft-cartodb/blob/master/config/environments/test.js.example)
+
+By default this image uses `development.js.example`, but you can choose a different file simply by specifying its name as the `command` to the container. Alternatively, you can supply your own configuration file (or files). To do so, extend this image by `COPY`ing a custom config file into `/Windshaft-cartodb/config/environments/`
+and pass the name of your file(s) as the `command` to the container. If you supply multiple filenames, the configuration objects they contain will be merged. Note that with respect to merging, files earlier in the list take precedence over files later in the list.
+
+Note also that we supply some additional configuration, in the file `base.config.js`, to make this image work together with the other `cartodb-docker` images. These configuration settings have the highest precedence, overriding the same fields in any other configuration files supplied. If you need to override these settings, therefore, you'll need to override the container's `entrypoint`.

postgis/README.md

@@ -0,0 +1,14 @@
+# cartodb-postgis
+### PostGIS database server for cartodb-docker
+
+`ihme/cartodb-postgis` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-postgis/
+
+This image contains a PostGIS database server with extensions specific to Carto, taken from the [cartodb-postgresql](https://github.com/CartoDB/cartodb-postgresql) GitHub repo. Its image tag references the corresponding version of that repo.
+
+## Data Persistence
+
+If you want to preserve the contents of the database beyond the life of the container itself (recommended), you'll need to mount a volume to the data directory (`/var/lib/postgresql/data` by default). The sample `docker-compose.yml` provided in this repo doesn't do so, because we don't know where you want to store the data.
+
+## Configuration
+
+There are several options for passing custom configuration to the server. The simplest option is to specify command-line flags to the `postgres` executable by passing them as the `command` to the container (`entrypoint` already invokes the executable itself via a startup script). We use the official [Postgres Docker image](https://hub.docker.com/_/postgres/) as a parent image. Consult the documentation for that image for details on how to further configure the server.

redis/README.md

@@ -0,0 +1,8 @@
+# cartodb-redis
+### Redis cache server for cartodb-docker
+
+`ihme/cartodb-redis` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-redis/
+
+This image lightly extends the [official Redis Docker image](https://hub.docker.com/_/redis/), configuring it to persist data on disk. Note, however, that you'll need to mount a volume on which to store this data (using the container path `/data`). The sample `docker-compose.yml` provided in this repo doesn't do so, because we don't know where you want to store the data.
+
+Further configuration can be passed to the Redis server by supplying a custom `command`. Existing configuration can be overriden by supplying a custom `entrypoint`.

router/README.md

@@ -0,0 +1,16 @@
+# cartodb-router
+### reverse proxy server for cartodb-docker
+
+`ihme/cartodb-router` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-router/
+
+This image extends the [official nginx Docker image](https://hub.docker.com/_/nginx/), configuring it to route Carto web traffic to the appropriate services. This simplifies requests, as all requests to Carto can use the same base URL and port.
+
+This server is not a part of Carto itself; it's just a router to make requests simpler. Feel free to modify it or replace it as your needs dictate. You could for instance add routing to your own custom web application that consumes Carto services. Or you could swap out this nginx server in favor of Varnish if you want an HTTP cache.
+
+## Routing
+
+| Service      | URL                  |
+| ------------ | -------------------- |
+| Maps API     | [baseURL]/api/v1/map |
+| SQL API      | [baseURL]/api/v2/sql |
+| Carto Editor | [baseURL]/           |

sqlapi/README.md

@@ -0,0 +1,19 @@
+# cartodb-sqlapi
+### SQL API for cartodb-docker
+
+`ihme/cartodb-sqlapi` image on DockerHub: https://hub.docker.com/r/ihme/cartodb-sqlapi/
+
+This image contains the [Carto SQL API](https://github.com/CartoDB/CartoDB-SQL-API), which allows making SQL queries against the PostGIS database via a REST API.
+
+## Configuration
+
+The server can be configured via a `.js` file in a specific format. Carto provides several such files as examples:
+- [development.js.example](https://github.com/CartoDB/CartoDB-SQL-API/blob/master/config/environments/development.js.example)
+- [production.js.example](https://github.com/CartoDB/CartoDB-SQL-API/blob/master/config/environments/production.js.example)
+- [staging.js.example](https://github.com/CartoDB/CartoDB-SQL-API/blob/master/config/environments/staging.js.example)
+- [test.js.example](https://github.com/CartoDB/CartoDB-SQL-API/blob/master/config/environments/test.js.example)
+
+By default this image uses `development.js.example`, but you can choose a different file simply by specifying its name as the `command` to the container. Alternatively, you can supply your own configuration file (or files). To do so, extend this image by `COPY`ing a custom config file into `/CartoDB-SQL-API/config/environments/`
+and pass the name of your file(s) as the `command` to the container. If you supply multiple filenames, the configuration objects they contain will be merged. Note that with respect to merging, files earlier in the list take precedence over files later in the list.
+
+Note also that we supply some additional configuration, in the file `base.config.js`, to make this image work together with the other `cartodb-docker` images. These configuration settings have the highest precedence, overriding the same fields in any other configuration files supplied. If you need to override these settings, therefore, you'll need to override the container's `entrypoint`.