Skip to content

Latest commit

 

History

History
185 lines (123 loc) · 7.58 KB

README.md

File metadata and controls

185 lines (123 loc) · 7.58 KB
icon description
truck-ramp-couch
Learn how to deploy and start the Router in Production.

Deployment

Run the Router

The router is a Go application provided as a self-contained Docker container. You can initiate the router by executing a single Docker command:

docker run \
    --name cosmo-router \
    -e GRAPH_API_TOKEN=<router_api_token> \
    -e LISTEN_ADDR=0.0.0.0:3002 \
    -p 3002:3002 \
    ghcr.io/wundergraph/cosmo/router:latest

You can generate a token with the wgc router token create CLI command.

Health checks

By default, the health check endpoint is exposed at the path /health. The health check reports if the router is alive.

Kubernetes

If you are on Kubernetes you can make use of our liveness and readiness checks. For convenience, we export both on /health/live and /health/ready:

containers:
  - name: router
    livenessProbe:
      httpGet:
        path: "/health/live"
        port: 3002
    readinessProbe:
      httpGet:
        path: "/health/ready"
        port: 3002

The liveness route returns 200 when the router is up. The readiness handler returns 200 only when the router is ready to serve requests.

Graceful shutdown

Before a Pod is terminated, a SIGTERM signal is sent to the Pod. Afterward, the Pod has 30 seconds to terminate itself (see terminationGracePeriodSeconds). Ensure that the router-specific timeouts are set below this threshold or increase the termination period appropriately.

For example, the following configuration was tested and could serve as a good starting point:

Router

  • GRACE_PERIOD_SECONDS=20
  • SHUTDOWN_DELAY_SECONDS=30

Pod

  • terminationGracePeriodSeconds=60

With that configuration, your router has 30 seconds to respond to all active connections before forcefully shutting down the server. Kubernetes will not terminate the container until the termination period of 60 seconds is over. Because the readiness endpoint no longer responds 200 and the container is inTERMINATION state any new traffic from being redirected to the instance is prevented.

GRACE_PERIOD_SECONDS becomes interesting when the router updates its schema. This is also the time the server has to gracefully shut down old clients before the switch is enforced.

{% hint style="info" %} For more information, we recommend the Kubernetes Best Practice Terminating With Grace article. {% endhint %}

Kubernetes & Container environments

To make the application reachable within the container network, you need to expose the router on 0.0.0.0 e.g LISTEN_ADDR=0.0.0.0:3002. This is also required when trying to access Prometheus metrics from outside the docker network. Use PROMETHEUS_LISTEN_ADDR="0.0.0.0:8088" to make the endpoint accessible.

Logging

Structured logging, also known as JSON logging, is enabled by default and can be controlled using the JSON_LOG environment variable. For development, we recommend setting this off because the logs are more friendly rendered.

Observability & Operations

The router exposes useful metrics about the process and memory. All metrics can be scraped from the Prometheus metrics endpoint. For more information please visit the dedicated Metrics section.

Tracing Sampling Rate

In scenarios with low traffic, a sampling rate of 1 (100%) is acceptable. However, for high-volume situations, we strongly recommend using a lower sampling rate. For instance, in a project experiencing 60 requests per second, a sampling rate of 0.1 (10% 6 req/s) is sufficient to generate valuable insights.

If you export to Cosmo Cloud, which is enabled by default, a sampling rate based on your subscription will be enforced. If your configured rate exceeds your account limit, a warning will be displayed. Please consider upgrading your plan or contact us for custom limits.

{% hint style="warning" %} Ensure that your subgraphs use parent-based sampling to inherit the sampling rate. For more information see OTEL instrumentation on Subgraphs. {% endhint %}

Configuration recommendations

Advanced Request Tracing (ART) is enabled by default; however, it will only be accessible if you have set DEV_MODE=true or GRAPH_API_TOKEN. In the latter case, we will exchange a public key from the control plane to ensure that only authorized requests from the Studio can access ART requests. This enables safe debugging of your router in production.

Therefore DEV_MODE=true should NOT be set when deploying your router to production.

Recommended resources

The specific resource recommendations depend on your constraints, but we have observed that low to medium-traffic projects benefit from the following setup:

  • 3 instances for high availability.
  • Each instance is equipped with 2 CPUs and 1GB of RAM.

Thanks to the utilization of Go and internal optimizations, the router demonstrates strong vertical scalability in relation to CPU and memory.

Custom Config

In Docker and Kubernetes, you can mount configuration files. In that way, you don't have to build a custom Docker image. By default, the router looks for a config at the location where it starts. This location is /config.yaml in our official Docker image. You can overwrite the path by setting the CONFIG_PATH environment variable.

Docker

Here we mount the router config from the host system. Take a closer look at -v flag.

docker run \
    --name cosmo-router \
    -e GRAPH_API_TOKEN=<router_api_token> \
    -e LISTEN_ADDR=0.0.0.0:3002 \
    -v ./config.yaml:/config.yaml \ # Mount the config in the same location as the binary
    -p 3002:3002 \
    ghcr.io/wundergraph/cosmo/router:latest

Kubernetes

In Kubernetes, we have the concepts of volumes and volumeMounts to realize the same behavior as in Docker. Below is a configuration example for the volume only.

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: router
          volumeMounts: # Mount the config in the same location as the binary
            - name: router-config
              mountPath: /config.yaml
              subPath: config.yaml
      volumes: # Create a volume from a configMap
        - name: router-config
          configMap:
            name: myConfigMap

Custom Image

In some scenarios, you have to provide a custom Dockerfile, e.g., when you need to build your custom Router binary. Here is an example of how the Dockerfile can look.

FROM --platform=${BUILDPLATFORM} golang:1.21 as builder

ARG TARGETOS
ARG TARGETARCH

ARG VERSION=dev
ENV VERSION=$VERSION

WORKDIR /app/

# Copy only the files required for go mod download
COPY go.* .

# Download dependencies
RUN go mod download

# Copy the rest of the files
COPY . .

# Run tests
RUN make test

# Build router
RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -trimpath -ldflags "-extldflags -static -X github.com/wundergraph/cosmo/router/core.Version=${VERSION}" -a -o router cmd/custom/main.go

FROM --platform=${BUILDPLATFORM} gcr.io/distroless/base-debian12

COPY --from=builder /app/router /router

CMD ["/router"]

EXPOSE 3002

If you are wondering about the BUILDPLATFORM, TARGETOS, TARGETARCH arguments, those are needed when building multi-arch images. You can remove them if you don't need them.