Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update concept documentation for 1.0 #2378

Merged
merged 5 commits into from
Sep 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions docs/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -334,41 +334,47 @@ pygmentsStyle = "friendly"
identifier = "architecture"
weight = 401
parent = "porter-architecture"
[[menu.main]]
name = "Security"
url = "/security-features/"
identifier = "porter-security"
weight = 402
parent = "porter-architecture"
[[menu.main]]
name = "CNAB"
url = "/cnab/"
identifier = "architecture-cnab"
weight = 402
weight = 412
parent = "porter-architecture"
[[menu.main]]
name = "Buildtime"
url = "/architecture-buildtime"
identifier = "architecture-buildtime"
weight = 403
weight = 413
parent = "porter-architecture"
[[menu.main]]
name = "Runtime"
url = "/architecture-runtime"
identifier = "architecture-runtime"
weight = 404
weight = 414
parent = "porter-architecture"
[[menu.main]]
name = "Invocation Images"
url = "/build-image"
identifier = "build-image"
weight = 405
weight = 415
parent = "porter-architecture"
[[menu.main]]
name = "Credentials"
url = "/credentials/"
identifier = "credentials"
weight = 406
weight = 416
parent = "porter-architecture"
[[menu.main]]
name = "Parameters"
url = "/parameters/"
identifier = "parameters"
weight = 407
weight = 417
parent = "porter-architecture"

# Operator
Expand Down
40 changes: 29 additions & 11 deletions docs/content/architecture-buildtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,39 @@ title: Porter Buildtime Architecture
description: Understanding Porter's Buildtime Architecture
---

This is a placeholder doc page. See our [contributing guide][contrib]
if you would like to add content for this page.
In Porter, "buildtime" refers to authoring and building a bundle.
A bundle is defined by a [manifest], porter.yaml, where you define:

# TODO
* [Parameters] to customize the bundle.
* [Credentials] to authenticate to services and resources used by the bundle.
* [Outputs] generated by the bundle.
* [Mixins] used by the bundle. Mixins help you install and interact with tools in the bundle.
* [Actions] such as install, upgrade or uninstall. These contain the steps to execute when that bundle action is run.

* Client binaries
* PORTER_HOME
* Dockerfile generation
* commands
* flow
Mixins must be installed to the same machine where the bundle is built.
So for example, if you want to use the kubernetes mixin in your bundle, first run `porter mixins install kubernetes`.
Mixins are cached in your PORTER_HOME directory, and the kubernetes mixin can be found in ~/.porter/mixins/kubernetes after it is installed.
Porter does not support specifying which version of a mixin to use in a bundle yet.
You can follow the [Mixins are Bundles proposal](https://github.com/getporter/proposals/blob/main/pep/005-mixins-are-bundles.md) to keep track of that feature's progress.

# Next
When the bundle author runs `porter build`, Porter first generates a Dockerfile to create an image for the bundle (known as the invocation image or installer).
You can define your own [custom Dockerfile](bundle/custom-dockerfile/) to customize and optimize the image.
The Dockerfile contains the following: a base image (debian), additional customizations generated by the mixins, and a COPY statement to include the files contained in the bundle directory where the porter.yaml file is located.
A copy of the Porter runtime, the mixin runtimes and the porter.yaml file are copied into the bundle.
After the image is built, Porter then generates a CNAB-compliant bundle.json file that defines the bundle's metadata.

* [Dependencies](/dependencies/)
# See Also

* [Create a bundle](/bundle/create/)
* [Use a custom Dockerfile](/bundle/custom-dockerfile/)
* [Mixin Architecture](/mixin-dev-guide/architecture/)
* [Building the Invocation Image](/build-image/)
* [Distribute a bundle](/distribute-bundles/)
* [Porter Runtime](/architecture-runtime/)

[contrib]: /contribute/guide/#documentation
[manifest]: /bundle/manifest/
[Parameters]: /parameters/
[Credentials]: /credentials/
[Outputs]: /bundle/manifest/#outputs
[Mixins]: /mixins/
[Actions]: /bundle/manifest/#bundle-actions
24 changes: 18 additions & 6 deletions docs/content/architecture-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,29 @@ title: Porter Runtime Architecture
description: Understanding Porter's Runtime Architecture
---

The Porter "runtime" refers to the embedded copy of the porter-runtime binary that is inside every single Porter bundle, located at /cnab/app/porter-runtime.
In Porter, "runtime" refers generally to the execution of a bundle.
The "Porter runtime" specifically refers to the embedded copy of the porter-runtime binary that is inside every single Porter bundle, located at /cnab/app/porter-runtime.

## Bundle Output
Below is an outline of what happens when a bundle is executed:

When a bundle is executed by Porter, the output generated by the Porter runtime is directed to STDERR.
Output generated by the mixins may be sent to either STDOUT or STDERR depending on the behavior of the tool that the mixin calls.
1. Porter identifies all parameters and credentials, resolving parameter and credentials sets against the host environment and any configured secret stores.
2. Porter runs a container using the bundle's invocation image, injecting the parameters and credentials as files and/or environment variables.
3. The entrypoint of the container is the Porter runtime. It parses the porter.yaml file embedded in the bundle, and executes the steps defined for the current action (such as install or upgrade).
4. For each step, Porter calls the corresponding mixins which are also embedded in the bundle.
5. The mixin performs an action, most commonly translating the yaml snippet for the step into a call to a development tool. For example, the helm3 mixin handles calling out to the helm CLI.
6. When the bundle is complete, Porter records the result of the run, and persists any outputs generated by the bundle.

## Bundle Console Output

When a bundle is executed by Porter, the console output generated by the Porter runtime is directed to STDERR.
Console output generated by the mixins may be sent to either STDOUT or STDERR depending on the behavior of the tool that the mixin calls.

For example, a bash script that uses "echo" will have its output sent to STDOUT.
Other mixins may send some output to STDOUT and some to STDERR depending on the message severity.

If you want to exclude Porter's output from your bundle output, do not pass the --debug flag to the corresponding porter command, such as porter install or porter upgrade.
If you want to exclude Porter's runtime console output, do not pass the --debug flag to the corresponding porter command, such as porter install or porter upgrade.
When a bundle is run in "debug mode", Porter will print additional output about how the bundle and mixins were run to STDERR.

[contrib]: /contribute/guide/#documentation
# See Also
* [Porter Buildtime](/architecture-buildtime/)

50 changes: 42 additions & 8 deletions docs/content/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,49 @@ title: Architecture
description: Detailed look at how Porter does its magic 🎩✨
---

TODO:
Porter is an implementation of the [Cloud Native Application Bundle](/cnab/) specification and creates installers, known as bundles, that understand how to install not only your application but its infrastructure and configuration.
A bundle helps you package up the logic for installing your application so that you can hand it off to another team, a customer, or just a co-worker who doesn't know all the ins and outs of your application and provide them a consistent installation experience that doesn't require them to know what tools you use to deploy or how the sausage is made.

* runtime v buildtime
* Our binaries and mixins
* Call flow
* How we implement CNAB, and where we add our own stuff outside of the spec (like dependencies)
* Oh yeah dependencies and buildtime v runtime
## Buildtime

A bundle author is responsible for understanding how to deploy an application and automate it inside a bundle using existing deployment tools, such as terraform or helm.
They use [mixins] to install tools into the bundle, and can include additional files like helm charts, kustomize files, or terraform modules.
In the porter.yaml file, they automate each step of the application's deployment:

* Collecting credentials
* Using parameters to customize the installation
* Creating infrastructure
* Setting up configuration files
* Installing software

The bundle author then builds the bundle into an OCI artifact which includes a docker image with the necessary files and development tools along with the bundle's metadata and any other docker images used by the bundle.
Once a bundle is built, it can be distributed using Docker / OCI registries.
This allows you to use existing tools and infrastructure to share your application with other teams, customers, and end-users.

Because the bundle contains everything you need to deploy, included referenced images, you can even [move a bundle into a disconnected or airgapped environment](/administrators/airgap/).
When written with airgap deployments in mind, a bundle can be deployed anywhere without requiring access to the original network or the internet.

Learn more about [how Porter works at buildtime](/architecture-buildtime/).

## Runtime

End users can then discover, inspect and run bundles with a consistent interface (porter) that doesn't rely on them understanding the underlying set of tools, scripts and architecture of the application.
Regardless of what is being installed, the commands look the same:

```
porter install --credential-set USER_CREDS --parameter-set CUSTOM_PARAMETERS --reference BUNDLE_REFERENCE
```

Once a bundle is installed, is tracked as an "installation" of the bundle.
An installation is a record of which bundle was installed, the parameters used to customize the installation, previous runs of the bundle, the current version of the bundle, along with other useful metadata.

Learn more about [how Porter works at runtime](/architecture-runtime/).

## See Also

* [Bundle Dependencies](/dependencies/)
* [Porter Build - Making Invocation Images](/build-image/)
* [Security Features](/security-features/)
* [Create a Bundle](/bundle/create/)
* [Distribute Bundles](/distribute-bundles/)
* [Airgapped Deployments](/administrators/airgap/)

[mixins]: /mixins/
45 changes: 43 additions & 2 deletions docs/content/bundle/custom-dockerfile.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,51 @@ By default, Porter uses the [1.4.0 dockerfile syntax](https://docs.docker.com/en
[buildkit]: https://docs.docker.com/develop/develop-images/build_enhancements/
[porter build]: /cli/porter_build/

# Bundles do not run as root

Porter runs the bundle image as a non-root user.
This means that if you need to initialize the user's home directory, you should use the [BUNDLE_USER](#BUNDLE_USER) build argument to locate the home directory.
If you need to run some commands as root and others as the non-root user that the bundle image will run under, you can use that same argument to switch the current user in the Dockerfile.
By default, all commands in the Dockerfile run as root (so that you can install and configure the image), so you will need to carefully use the USER statement to switch to the non-root user.

Here is a truncated and commented example of how the helm3 mixin performs some setup as root and some as the non-root user:
```Dockerfile
# Truncated, above we set up the image...

# The helm3 mixin performs system-level configuration, such as installing the helm cli
ENV HELM_EXPERIMENTAL_OCI=1
RUN apt-get update && apt-get install -y curl
RUN curl https://get.helm.sh/helm-v3.8.2-linux-amd64.tar.gz --output helm3.tar.gz
RUN tar -xvf helm3.tar.gz && rm helm3.tar.gz
RUN mv linux-amd64/helm /usr/local/bin/helm3
RUN curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.22.1/bin/linux/amd64/kubectl &&\
mv kubectl /usr/local/bin && chmod a+x /usr/local/bin/kubectl

# The helm3 mixin switches temporarily to run a couple commands as the non-root user
USER ${BUNDLE_USER}
RUN helm3 repo add stable https://charts.helm.sh/stable
RUN helm3 repo update

# The helm3 mixin switches back to root so that the rest of the Dockerfile is run with a user with the correct permissions
USER root

# Truncated, continue setting up the bundle's image...
```

The non-root user matters both at build time when the image is built and also at runtime when it runs in a container.
The user that the container runs as is not configurable and any files written by the container are owned by the non-root user.
This is relevant when running a bundle in a Kubernetes pod (either with the kubernetes driver manually or with the Porter Operator).
It is important when mounting volumes into the pod that the non-root user has read/write access.

# Special Comments
Porter uses comments as placeholders to inject lines into your Dockerfile that all Porter bundles require.
You can move the comment to another location in the file to optimize your Docker build times and layer caching.
If you omit the comment entirely, Porter will still inject the contents for that section into your Dockerfile and we recommend keeping the comments in so that you can control where the contents are injected.
If you omit the comment entirely, Porter will still inject the contents for that section into your Dockerfile, and we recommend keeping the comments in so that you can control where the contents are injected.

## PORTER_INIT

Porter includes additional Dockerfile lines that standardize all Porter bundles, such as declaring the BUNDLE_DIR argument, and creating a user for the bundle to run as. You can control where these lines are injected by placing a comment in your Dockerfile temlate:
Porter includes additional Dockerfile lines that standardize all Porter bundles, such as declaring the BUNDLE_DIR argument, and creating a user for the bundle to run as.
You can control where these lines are injected by placing a comment in your Dockerfile template:

```Dockerfile
# PORTER_INIT
Expand Down Expand Up @@ -136,6 +173,10 @@ If you do, you are responsible for setting the file permissions so that the bund
COPY . ${BUNDLE_DIR}
```

## See Also

* [Why you do not want to run containers as root](https://medium.com/@mccode/processes-in-containers-should-not-run-as-root-2feae3f0df3b)
* [Security Features](/security-features/)

[Buildkit]: https://docs.docker.com/develop/develop-images/build_enhancements/
[experimental]: /configuration/#experimental-feature-flags
Expand Down
1 change: 1 addition & 0 deletions docs/content/docker-images/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ description: The Porter Operator agent image

The [ghcr.io/getporter/porter-agent][porter-agent] Docker image is intended for use by the [Porter Operator] which runs on Kubernetes.
If you need to run Porter in a local container, not on Kubernetes, you should use the [porter client] image.
The porter agent is also available in the [PlatformOne IronBank registry](https://registry1.dso.mil/harbor/projects/3/repositories/opensource%2Fgetporter%2Fporter-agent/artifacts-tab).

It has tags that match what is available from our [install](/install/) page: latest, canary and specific versions such as v0.38.1.

Expand Down
1 change: 1 addition & 0 deletions docs/content/docker-images/workshop.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Porter Workshop Docker Image
description: How to use the getporter/workshop Docker image
draft: true
---

The [ghcr.io/getporter/workshop][workshop] Docker image provides the Porter client installed in a
Expand Down
14 changes: 0 additions & 14 deletions docs/content/finding-bundles.md

This file was deleted.

Loading