Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add store and hook documentation #938

Merged
merged 2 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ By understanding these challenges, teams can better prepare for a smooth transit
## Brownfield Development in Atmos

Atmos is easier for new organizations or "greenfield" environments because you need to architect Terraform according to
our [best practices](/best-practices/components) to get all the benefits of Atmos. For example, when using our [Terraform components](https://github.com/cloudposse/terraform-aws-components), we frequently use [Terraform Remote State](/core-concepts/components/terraform/remote-state) to retrieve the outputs from other components.
our [best practices](/best-practices/components) to get all the benefits of Atmos. For example, when using our [Terraform components](https://github.com/cloudposse/terraform-aws-components), we frequently use [Terraform Remote State](/core-concepts/share-data/remote-state) to retrieve the outputs from other components.

This works well when you use our components but less so when you operate in a "brownfield" environment, for example,
with an existing VPC, S3 bucket, or IAM role.
Expand Down Expand Up @@ -56,7 +56,7 @@ The `vpc` Terraform component needs the outputs from the `vpc-flow-logs-bucket`
configure [VPC Flow Logs](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html).

Let's redesign the example with the `vpc` and `vpc-flow-logs-bucket` components described in
[Terraform Component Remote State](/core-concepts/components/terraform/remote-state) and configure the `static` remote state for
[Terraform Component Remote State](/core-concepts/share-data/remote-state) and configure the `static` remote state for
the `vpc-flow-logs-bucket` component to use an existing S3 bucket.

## Examples
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ id: state-backend
import Intro from '@site/src/components/Intro'

<Intro>
Atmos supports configuring [Terraform Backends](/core-concepts/components/terraform/backends) to define where Terraform stores its [state](https://developer.hashicorp.com/terraform/language/state), and [Remote State](/core-concepts/components/terraform/remote-state) to get the outputs of a [Terraform component](/core-concepts/components), provisioned in the same or a different [Atmos stack](/core-concepts/stacks), and use the outputs as inputs to another Atmos component.
Atmos supports configuring [Terraform Backends](/core-concepts/components/terraform/backends) to define where Terraform stores its [state](https://developer.hashicorp.com/terraform/language/state), and [Remote State](/core-concepts/share-data/remote-state) to get the outputs of a [Terraform component](/core-concepts/components), provisioned in the same or a different [Atmos stack](/core-concepts/stacks), and use the outputs as inputs to another Atmos component.
</Intro>

Bear in mind that Atmos is simply managing the configuration of the Backend;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ Initial Atmos configuration can be controlled by these environment variables:

## Special Considerations for Terraform Components

If you are relying on Atmos discovering the `atmos.yaml` based on your current working directory (e.g. at the root of repository), it will work for the `atmos` CLI; however, it will **not work** for [Component Remote State](/core-concepts/components/terraform/remote-state) because it uses the [terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils) Terraform provider.
If you are relying on Atmos discovering the `atmos.yaml` based on your current working directory (e.g. at the root of repository), it will work for the `atmos` CLI; however, it will **not work** for [Component Remote State](/core-concepts/share-data/remote-state) because it uses the [terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils) Terraform provider.

This is because Terraform executes provider from the component's folder (e.g. `components/terraform/vpc`), so it will no longer find the file in the root of the repository, since the working directory has changed.

Expand Down
117 changes: 117 additions & 0 deletions website/docs/core-concepts/projects/configuration/stores.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
title: Configure Stores
sidebar_position: 4
sidebar_label: Configure Stores
id: stores
---
import Intro from '@site/src/components/Intro'

<Intro>
Atmos supports the concept of remote stores to facilitate the sharing of values between components or between
some external process and a component. In `atmos`, values are saved to stores via
[hooks](/core-concepts/stacks/hooks) and are read using the [!store](/core-concepts/stacks/yaml-functions/store.mdx)
yaml function. Values can also be saved to stores from outside of `atmos`, for example, from a CI/CD pipeline or a
script.

Currently, the following stores are supported:

- [AWS SSM Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html)
- [Artifactory](https://jfrog.com/artifactory/)
</Intro>

Atmos stores are configured in the `atmos.yaml` file and available to use in stacks via the
[store](/core-concepts/stacks/yaml-functions/store) YAML function.

## CLI Configuration

All of these settings should be configured in the [Atmos CLI Configuration](/cli/configuration) found in `atmos.yaml`.

### Artifactory

```yaml
stores:
dev/artifactory:
type: artifactory
options:
url: https://mydevartifactory.jfrog.io/artifactory
repo_name: tfsharedstore

prod/artifactory:
type: artifactory
options:
url: https://myprodartifactory.jfrog.io/artifactory
repo_name: tfsharedstore
access_token: !env PROD_JFROG_ACCESS_TOKEN
```

<dl>
<dt>`stores.[store_name]`</dt>
<dd>This map key is the name of the store. It must be unique across all stores. This is how the store is referenced in the `store` function.</dd>

<dt>`stores.[store_name].type`</dt>
<dd>Must be set to `artifactory`</dd>

<dt>`stores.[store_name].options`</dt>
<dd>A map of options specific to the store type. For Artifactory, the following options are supported:</dd>

<dt>`stores.[store_name].options.access_token (optional)`</dt>
<dd>An access token to use for authentication. This is not recommended as it is less secure than using the
`JFROG_ACCESS_TOKEN` or `ARTIFACTORY_ACCESS_TOKEN` environment variables. See [Authentication](#authentication) below
mcalhoun marked this conversation as resolved.
Show resolved Hide resolved
for more information.</dd>

<dt>`stores.[store_name].options.prefix (optional)`</dt>
<dd>A prefix path that will be added to all keys stored or retreived from SSM Parameter Store. For example if the prefix
is `/atmos/infra-live/`, and if the stack is `plat-us2-dev`, the component is `vpc`, and the key is `vpc_id`, the full path
would be `/atmos/infra-live/plat-us2-dev/vpc/vpc_id`.</dd>

<dt>`stores.[store_name].options.repo_name (required)`</dt>
<dd>The name of the Artifactory repository to use.</dd>

<dt>`stores.[store_name].options.url (required)`</dt>
<dd>The URL of the Artifactory instance.</dd>
</dl>

#### Authentication

The Artifactory store supports using an access token for authentication. The access token can be set directly in the
`atmos.yaml` or via the `JFROG_ACCESS_TOKEN` or `ARTIFACTORY_ACCESS_TOKEN` environment variables.

NOTE: It is not recommended to set the access token in plain text in `atmos.yaml` as it is not secure. Specifying the
access token via config is allowed so that you can support the use case where you have multiple Artifactory stores
in `atmos.yaml` and you need to specify different access tokens for each. In this case, you can use the `!env`
YAML function to set the access token from an environment variable.

### AWS SSM Parameter Store

```yaml
stores:
prod/ssm:
type: aws-ssm-parameter-store
options:
region: us-east-2
```

<dl>
<dt>`stores.[store_name]`</dt>
<dd>This map key is the name of the store. It must be unique across all stores. This is how the store is referenced in the `store` function.</dd>

<dt>`stores.[store_name].type`</dt>
<dd>Must be set to `aws-ssm-parameter-store`</dd>

<dt>`stores.[store_name].options`</dt>
<dd>A map of options specific to the store type. For AWS SSM Parameter Store, the following options are supported:</dd>

<dt>`stores.[store_name].options.prefix (optional)`</dt>
<dd>A prefix path that will be added to all keys stored or retreived from SSM Parameter Store. For example if the prefix
is `/atmos/infra-live/`, and if the stack is `plat-us2-dev`, the component is `vpc`, and the key is `vpc_id`, the full path
would be `/atmos/infra-live/plat-us2-dev/vpc/vpc_id`.</dd>

<dt>`stores.[store_name].options.region (required)`</dt>
<dd>The AWS region to use for the SSM Parameter Store.</dd>
</dl>

#### Authentication

The AWS SSM Parameter Store supports the standard AWS methods for authentication and the `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` environment variables.

43 changes: 40 additions & 3 deletions website/docs/core-concepts/share-data/share-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,15 @@ import File from '@site/src/components/File'
import Note from '@site/src/components/Note'

<Intro>
Breaking apart your infrastructure into loosely coupled components is a great way to manage complexity and reuse code. However, sometimes, you need to share data between components. In Atmos, you can easily share settings, configurations, and outputs between components and even tap into external data sources.
Breaking up your infrastructure components into loosely coupled components is a great way to manage complexity and
reuse code. However, these smaller components often lead to a situation where you need to share data between components.
In Atmos, there are several ways you can easily share settings, configurations, and outputs among components and even
tap into external data sources and stores.
</Intro>

There are two ways to approach this: using native Terraform support for remote state to read outputs from other components or using template functions in stack configurations. In this chapter, you’ll learn how to share state between components within the same stack or even across different stacks.
There are multiple ways to approach this: using native Terraform support for remote state to read outputs from other
components or using template functions in stack configurations. In this chapter, you’ll learn how to share state between
components within the same stack or even across different stacks.

<KeyPoints>
- Why you might need to share data between components
Expand All @@ -26,6 +31,38 @@ There are two ways to approach this: using native Terraform support for remote s
</KeyPoints>


## Using YAML Functions

### Function: `!store`

The `!store` YAML function can read data from a remote store such as SSM Parameter Store or Artifactory.

For example, we can read the `vpc_id` output of the `vpc` component in the current stack from the SSM Parameter Store
configured in `atmos.yaml` as 'ssm/prod` simply by doing:

```yaml
components:
terraform:
cluster:
vars:
vpc_id: !store ssm/prod vpc vpc_id
```

To access the configuration of a component in a different stack, you can specify the stack name as the third argument.
For example, here we're reading the `vpc_id` output of the `vpc` component in the `staging` stack:

```yaml
components:
terraform:
cluster:
vars:
vpc_id: !store ssm/prod vpc staging vpc_id
```

<ActionCard>
For more advanced examples, check out the `!store` yaml function documentation.
<PrimaryCTA to="/core-concepts/stacks/yaml-functions/store">Learn More</PrimaryCTA>
</ActionCard>

## Using Template Functions

Expand Down Expand Up @@ -173,5 +210,5 @@ resource "aws_network_acl" "default" {

<ActionCard>
Use the Terraform-native `remote-state` module to share data between components.
<PrimaryCTA to="/core-concepts/components/terraform/remote-state">Learn How</PrimaryCTA>
<PrimaryCTA to="/core-concepts/share-data/remote-state">Learn How</PrimaryCTA>
</ActionCard>
70 changes: 70 additions & 0 deletions website/docs/core-concepts/stacks/hooks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: Manage Lifecycle Events with Hooks
sidebar_position: 6
sidebar_label: Manage Lifecycle Events with Hooks
description: Use lifecycle hooks to take action at various points in the lifecycle of your components.
id: hooks
---
import Terminal from '@site/src/components/Terminal'
import Intro from '@site/src/components/Intro'
import File from '@site/src/components/File'

<Intro>
Atmos supports the ability to take action at various points in the lifecycle of your components. This is done by
configuring the `hooks` section in your stack manifest for the component that you want to take action on.
</Intro>


## Hooks Schema

The `hooks` section schema is as follows:

```yaml
components:
terraform:
top-level-component1:
hooks:
store-outputs:
events:
- after-terraform-apply
command: store
name: prod/ssm
outputs:
vpc_id: .id
```

## Supported Lifecycle Events

Atmos supports the following lifecycle events:

- `after-terraform-apply` (this event is triggered after the `atmos terraform apply` or `atmos terraform deploy` command is run)

## Supported Commands

## store

The `store` command is used to write data to a remote store.

<dl>
<dt>`hooks.[hook_name]`</dt>
<dd>This map key is the name you want to give to the hook. This must be unique for each hook in the component.</dd>

<dt>`hooks.[hook_name].events`</dt>
<dd>
This is a list of [Supported Lifecycle Events](#supported-lifecycle-events) that should trigger running the command.
</dd>

<dt>`hooks.[hook_name].command`</dt>
<dd>Must be set to `store`</dd>

<dt>`hooks.[hook_name].name`</dt>
<dd>The name of the store to use.</dd>

<dt>`hooks.[hook_name].outputs`</dt>
<dd>
A map of values that will be written to the store under the key for this component. The key is the name of the key in
the store. The value is the value to write to the store. If the value begins with a dot (`.`), it will be treated as a
[Terraform output](https://developer.hashicorp.com/terraform/language/values/outputs) and the value will be retrieved
from the Terraform state for the current component.
</dd>
</dl>
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Atmos component in a stack, and use it in `Go` templates in Atmos component conf
<dd>Using the `outputs` section in the `atmos.Component` command is an convenient way to read the outputs (remote state)
of a component in a stack directly in Atmos stack manifests. It is an alternative to using the `remote-state`
module and configuring Terraform/OpenTofu components to use the `remote-state` module as described in
<a href="/core-concepts/components/terraform/remote-state">Component Remote State</a></dd>
<a href="/core-concepts/share-data/remote-state">Component Remote State</a></dd>

<dt>`attribute`</dt>
<dd>
Expand Down
Loading
Loading