Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates the dockerfile setup #19

Merged
merged 4 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Ignore .git dirs
.git
.github

# Ignore tests directory
**/tests/*
**/test_*.py

# Ignore docs directory
**/docs/*

# Ignore Python bytecode
**/__pycache__/
**/*.pyc
52 changes: 52 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Stage 1: Development stage for Python dependencies
FROM python:3.10-slim as dep-stage

# Set up app dir
WORKDIR /tmp

# Copy project files over
COPY ./pyproject.toml ./poetry.lock ./

# Install build-essential package
RUN apt-get update && \
apt-get install -y \
build-essential \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Install poetry and project dependencies
RUN pip install poetry && \
poetry config virtualenvs.create false && \
poetry install --no-root && \
rm -rf ~/.cache

# Stage 2: Development stage for the project
FROM dep-stage as dev-stage

# Copy the main project files over and install
COPY ./ ./
RUN poetry install --only main

# Create dir for socket and logs
RUN mkdir -p /tmp/webapp

# Setting environment variables
# these can be manually overridden
ENV MODULE_NAME="valis.wsgi"
ENV VALIS_SOCKET_DIR='/tmp/webapp'
ENV VALIS_LOGS_DIR='/tmp/webapp'
ENV VALIS_ALLOW_ORIGIN="http://localhost:8080"
ENV VALIS_DB_REMOTE=True

# Stage 3: Build stage (inherits from dev-stage)
FROM dev-stage as build-stage

# Set a label
LABEL org.opencontainers.image.source https://github.com/sdss/valis
LABEL org.opencontainers.image.description "valis production image"

# Expose the port
EXPOSE 8000

# Start the FastAPI app for production
CMD ["poetry", "run", "gunicorn", "-c", "python/valis/wsgi_conf.py", "valis.wsgi:app"]
49 changes: 49 additions & 0 deletions Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Stage 1: Development stage for Python dependencies
FROM python:3.10-slim as dep-stage

# Set up app dir
WORKDIR /tmp

# Copy project files over
COPY ./pyproject.toml ./poetry.lock ./

# Install build-essential package
RUN apt-get update && \
apt-get install -y \
build-essential \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Install poetry and project dependencies
RUN pip install poetry && \
poetry config virtualenvs.create false && \
poetry install --no-root && \
rm -rf ~/.cache

# Stage 2: Development stage for the project
FROM dep-stage as dev-stage

# Copy the main project files over and install
COPY ./ ./
RUN poetry install --only main

# setting environment variables
# these can be manually overridden
ENV MODULE_NAME="valis.wsgi"
ENV VALIS_SOCKET_DIR='/tmp/valis'
ENV VALIS_LOGS_DIR='/tmp'
ENV VALIS_ALLOW_ORIGIN="http://localhost:3000"
ENV VALIS_DB_REMOTE=True

# Stage 3: Build stage (inherits from dev-stage)
FROM dev-stage as build-stage

# Set a label
LABEL org.opencontainers.image.source https://github.com/sdss/valis
LABEL org.opencontainers.image.description "valis development image"

# Expose port 8000
EXPOSE 8000

# Start the FastAPI app with hot-reloading for development
CMD ["uvicorn", "valis.wsgi:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
62 changes: 49 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,23 +81,59 @@ db_user: {unid}


## Deployment
### Deploying at Utah in api.sdss.org docker
- Login to `lore` or `manga` machines
- Login to the api.sdss.org docker with `ssh -p 2209 [unid]@api.sdss.org`
- If needed, load valis module with `ml load valis`
- cd to `$VALIS_DIR`
- run `run_valis` alias or `poetry run gunicorn -c python/valis/wsgi_conf.py valis.wsgi:app`

This section describes a variety of deployment methods. Valis uses gunicorn as its
wsgi http server. It binds the app both to port 8000, and a unix socket. The defaut mode
is to start valis with an awsgi uvicorn server, with 4 workers.

### Deploying Zora + Valis together
See the SDSS [Zora+Valis Docker](https://github.com/sdss/zora_valis_dockers) repo page.

### Deploying at Utah in dataviz-dm
TBD

### Running manually via gunicorn + nginx
- Setup a local nginx server with a /valis location
- export VALIS_SOCKET_DIR=/tmp/valis
- run `gunicorn -c python/valis/wsgi_conf.py valis.wsgi:app`
### Running Docker Compose
This builds and sets up the valis docker running with nginx, mapped to a unix socket at `unix:/tmp/valis/valis.sock`. It binds the internal nginx port 8080 to localhost port 5000.

- Navigate to `docker` folder
- Run `docker-compose -f docker-compose.yml build` to build the docker images
- Run `docker-compose -f docker-compose.yml up -d` to start the containers in detached mode
- Navigate to `http://localhost:5000/valis`
- To stop the service and remove containers, run `docker-compose -f docker-compose.yml down`
This also exposes valis to port 8000, and should be available at `http://localhost:8000`.

### Running the Docker

There are two dockerfiles, one for running in development mode and one for production. To connect valis to the `sdss5db` database, you'll need to set several **VALIS_DB_XXX** environment variables during the `docker run` command.

- VALIS_DB_REMOTE: Set this to True
- VALIS_DB_HOST: the database host machine
- VALIS_DB_USER: the database user
- VALIS_DB_PASS: the database password
- VALIS_DB_PORT: the database port

The following examples show how to connect the valis docker to a database running on the same machine, following the database setup instructions above.

**Development**

To build the docker image, run

`docker build -t valis-dev -f Dockerfile.dev .`

To start a container, run
```bash
docker run -p 8000:8000 -e VALIS_DB_REMOTE=True -e VALIS_DB_HOST=host.docker.internal -e VALIS_DB_USER=[user] -e VALIS_DB_PASS=[password] -e VALIS_DB_PORT=6000 valis-dev
```

**Production**

To build the docker image, run

`docker build -t valis -f Dockerfile .`

To start a container, run
```bash
docker run -p 8000:8000 -e VALIS_DB_REMOTE=True -e VALIS_DB_HOST=host.docker.internal -e VALIS_DB_USER=[user] -e VALIS_DB_PASS=[password] -e VALIS_DB_PORT=6000 valis
```
Note: If your docker vm has only a small resource allocation, the production container may crash on start, due to the number of workers allocated. You can adjust the number of workers with the `VALIS_WORKERS` envvar. For example, add `-e VALIS_WORKERS=2` to scale the number of workers down to 2.

### Podman

All dockerfiles work with `podman`, and the syntax is the same as above. Simply replace `docker` with `podman`.
28 changes: 0 additions & 28 deletions docker/docker-compose.yml

This file was deleted.

3 changes: 0 additions & 3 deletions docker/nginx/Dockerfile

This file was deleted.

36 changes: 0 additions & 36 deletions docker/nginx/nginx.conf

This file was deleted.

34 changes: 0 additions & 34 deletions docker/valis/Dockerfile

This file was deleted.

6 changes: 0 additions & 6 deletions docker/valis/docker_prestart.sh

This file was deleted.

Loading