v1.153.1

Skipping disabled components in `atmos describe affected` @shirkevich (#942)

## what

• Skipping disabled components in atmos describe affected

why

• Components marked with metadata.enabled: false should be excluded from the affected components list
• Improves accuracy of the describe affected command by only showing components that would actually be processed

v1.153.0

Introduce Atmos YAML functions `!include` and `!env` @aknysh (#943)

what

• Introduce Atmos YAML functions !include and !env
• Add docs
  • https://atmos.tools/core-concepts/stacks/yaml-functions/env/
  • https://atmos.tools/core-concepts/stacks/yaml-functions/include/

why

• The !env YAML function is used to retrieve environment variables and assign them to the sections in Atmos stack manifests

• The !include YAML function allows downloading local or remote files from different sources, and assigning the file contents or individual values to the sections in Atmos stack manifests

description

!env Atmos YAML function

The !env Atmos YAML function is used to retrieve environment variables and assign them to the sections in Atmos stack manifests.

  # Get the value of an environment variable.
  # If the environment variable is not present in the environment, `null` will be assigned
  !env <env-var-name>

  # Get the value of an environment variable.
  # If the environment variable is not present in the environment, the `default-value` will be assigned
  !env <env-var-name> <default-value>

If the function is called with one argument (the name of the environment variable), and the environment variable is
not present, null will be assigned to the corresponding section in the Atmos manifest.

If the function is called with two arguments (the name of the environment variable and the default value), and the
environment variable is not present, the default value will be assigned to the corresponding section in the Atmos manifest.

Examples:

vars:
  # `api_key` will be set to `null` if the environment variable `API_KEY` is not present in the environment
  api_key: !env API_KEY
  # `app_name` will be set to the default value `my-app` if the environment variable `APP_NAME` is not present in the environment
  app_name: !env APP_NAME my-app

settings:
  # `provisioned_by_user` will be set to `null` if the environment variable `ATMOS_USER` is not present in the environment
  provisioned_by_user: !env ATMOS_USER

Handling default values with spaces:

If you need to provide default values with spaces, enclose them in double quotes and use single quotes around the whole expression.

For example:

  # `app_name` will be set to the default value `my app` if the environment variable `APP_NAME` is not present in the environment
  app_name: !env 'APP_NAME "my app"'

  # `app_description` will be set to the default value `my app description` if the environment variable `APP_DESCRIPTION` is not present in the environment
  app_description: !env 'APP_DESCRIPTION "my app description"'

!include Atmos YAML function

The !include Atmos YAML function allows downloading local or remote files from different sources, and assigning the file contents or individual values to the sections in Atmos stack manifests.

The !include function can be called with either one or two parameters:

  # Download the file and assign its content to the variable
  !include <file-path>

  # Download the file, filter the content using the YQ expression,
  # and assign the result to the variable
  !include <file-path> <yq-expression>

Examples:

components:
  terraform:
    my-component:
      vars:
        # Include a local file with the path relative to the current Atmos manifest
        values: !include ./values.yaml
        # Include a local file with the path relative to the current Atmos manifest and query the `vars.ipv4_primary_cidr_block` value from the file using YQ
        ipv4_primary_cidr_block: !include ./vpc_config.yaml .vars.ipv4_primary_cidr_block
        # Include a local file relative to the `base_path` setting in `atmos.yaml`
        vpc_defaults: !include stacks/catalog/vpc/defaults.yaml
        # Include a local file in HCL format
        hcl_values: !include ./values.hcl
        # Include a local file in HCL format with Terraform variables
        tfvars_values: !include ../components/terraform/vpc/vpc.tfvars
        # Include a local Markdown file
        description: !include ./description.md
        # Include a local text file
        text: !include a.txt
        # Include a local text file with spaces in the file name
        text2: !include '"my config.txt"'
        # Include a local text file on Windows with spaces in the file name, and get the `config.tests` value from the file
        tests: !include '"~/My Documents/dev/tests.yaml" .config.tests'
        # Download and include a remote YAML file using HTTPS protocol, and query the `vars` section from the file
        region_values: !include https://raw.githubusercontent.com/cloudposse/atmos/refs/heads/main/examples/quick-start-advanced/stacks/mixins/region/us-east-2.yaml .vars
        # Download and include a remote JSON file and query the `api` section from the file
        allowed_ips: !include https://api.github.com/meta .api
      settings:
        config:
          # Include a local JSON file and query the `user_id` variable from the file
          user_id: !include ./user_config.json .user_id

Description:

The YAML standard provides anchors and aliases, that allow you
to reuse and reference pieces of your YAML file, making it more efficient and reducing duplication.

Atmos supports YAML anchors and aliases, but the biggest limitation is that they are only available within the file in
which they are defined. You cannot reuse anchors across different files.

The !include Atmos YAML function overcomes this limitation by allowing you to include the content or individual values
from different local and remote sources. The !include function also provides the following features:

• Supports local files with absolute and relative paths

• Supports the remote protocols provided by the go-getter library

• Allows you to use YQ expressions to query and filter the content of the files to retrieve individual values

• Automatically detects the format of the files regardless of the file extensions. It supports files in JSON, YAML and HCL (tfvars) formats, and automatically converts them into correct YAML structures (simple and complex types like maps and lists are supported). All other files are returned unchanged, allowing you, for example, to include text and Markdown files as strings in Atmos manifests

Supported File Protocols:

The !include function supports the following local file paths:

• absolute paths (e.g. /Users/me/Documents/values.yaml)
• paths relative to the current Atmos manifest where the !include function is executed (e.g. ./values.yaml, ../config/values.yaml)
• paths relative to the base_path defined in atmos.yaml CLI config file (e.g. stacks/catalog/vpc/defaults.yaml)

To download remote files from different sources, the !include function uses go-getter
(used by Terraform for downloading modules) and supports the following protocols:

• tar - Tar files, potentially compressed (tar.gz, tar.bz2, etc.)
• zip - Zip files
• http - HTTP URLs
• https - HTTPS URLs
• git - Git repositories, which can be accessed via HTTPS or SSH
• hg - Mercurial repositories, accessed via HTTP/S or SSH
• s3 - Amazon S3 bucket URLs
• gcs - Google Cloud Storage URLs
• oci - Open Container Initiative (OCI) images
• scp - Secure Copy Protocol for SSH-based transfers
• sftp - SSH File Transfer Protocol

Using YQ Expressions to retrieve individual values from files:

To retrieve individual values from complex types such as maps and lists, or do any kind of filtering or querying,
you can utilize YQ expressions.

For example:

• Retrieve the first item from a list

subnet_id1: !include <file-path> .private_subnet_ids[0]

• Read a key from a map

username: !include <file-path> .config_map.username

For more details, review the following docs:

• YQ Guide
• YQ Recipes

Handling file paths and YQ expressions with spaces:

If you have spaces in the file names or the YQ expressions, enclose the file path and YQ expression in double quotes and
the whole expression in single quotes.

For example, on Windows:

  vars:
    values: !include '"~/My Documents/dev/values.yaml"'
    config: !include '"~/My Documents/dev/config.json" "<yq-expression-with-spaces>"'

On macOS and Linux:

  vars:
    values: !include './values.yaml "<yq-expression-with-spaces>"'
    description: !include '"component description.md"'

v1.152.1

Update `editorconfig` to version 3.1.2 @samtholiya (#955)

what

• Update editorconfig library to latest v3.1.2

why

• Supports many new features mentioned in release notes here

v1.152.0

Standardize Help and Usage content for all commands and sub commands @samtholiya (#914)

what

• Standardize Help and Usage content for all commands and sub commands
• Every command has intuitive display help and usage

why

• More consistent command-line interface with improved error handling and messaging

v1.151.0

Add `.editorconfig` validation @samtholiya (#896)

what

• Added functionality to atmos validate to validate against the .editorconfig file if it is defined
• Integrated the editorconfig-checker library to perform the validation
• Updated docs

why

• The change ensures that the .editorconfig file is validated, which helps maintain consistency in code formatting across teams.
• By incorporating editorconfig-checker, the validation process becomes more comprehensive, ensuring that both OPA/JSON schemas and editor configuration are correct
• This update was made to improve development workflow by catching formatting issues early during the validation process

Add Support for Testing Golden Snapshots @osterman (#939)

what

• Generate snapshots locally & commit them with the -regenerate-snapshots argument
• Test for changes by comparison golden snapshots with output
• Support TTY emulation (pty)
• Add support for !not tag to negate conditions
• Add a JSON schema for the test cases (to avoid silent failures)

why

• Too hard to add thorough tests for command line invocations

v1.150.0

Add logs level validation on the command line @Cerebrovinny (#946)

what

• Add logs level validation on the command line, e.g atmos describe affected --logs-level=Trace

why

• Validate the logs level specified in the --logs-level flag in addition to logs level configured in atmos.yaml or set in the environment variable ATMOS_LOGS_LEVEL, e.g. ATMOS_LOGS_LEVEL=Trace atmos terraform plan vpc --stack dev

v1.149.0

Update `!terraform.output` and `atmos.Component` functions @aknysh (#944)

what

• Improve !terraform.output YAML function
• Improve atmos.Component template function

why

• When executing !terraform.output and atmos.Component functions, honor the components.terraform.init_run_reconfigure setting in atmos.yaml. If it's set to true, add the -reconfigure flag to the terraform output command when executing it to get the component outputs. This prevents an issue when you use the functions to retrieve the outputs of the same component from many different stacks (without using the -reconfigure flag, Terraform detects that the backend configuration has changed and complains about it)

• Use YQ expressions to retrieve items from complex output types

  To retrieve items from complex output types such as maps and lists, or do any kind of filtering or querying,
  you can utilize YQ expressions.

  For example:

  • Retrieve the first item from a list

  subnet_id1: !terraform.output vpc .private_subnet_ids[0]

  • Read a key from a map

  username: !terraform.output config .config_map.username

  Examples

  components:
    terraform:
      my_lambda_component:
        vars:
          vpc_config:
            # Output of type list
            subnet_ids: !terraform.output vpc private_subnet_ids
            # Use a YQ expression to get an item from the list
            subnet_id1: !terraform.output vpc .private_subnet_ids[0]
            # Output of type map
            config_map: !terraform.output config {{ .stack }} config_map
            # Use a YQ expression to get a value from the map
            username: !terraform.output config .config_map.username

references

• https://github.com/mikefarah/yq
• https://mikefarah.gitbook.io/yq
• https://mikefarah.gitbook.io/yq/recipes
• https://mikefarah.gitbook.io/yq/operators/pipe

v1.148.1

add stores and hooks

## what

Atmos now supports the ability to use stores to share data among components as well as external sources.

• Add the concept of stores for reading shared data
• Add the concept of hooks for writing data to stores after terraform apply
• Add a new store for artifactory
• Add a new store for AWS SSM Parameter Store

why

When decomposing large terraform modules (terraliths) into smaller components with a smaller blast radius, it often brings a need to share data (outputs) of one component in another component. Previously atmos supported the !terraform.output and other data sources for this purpose, but the performance and security context made this task difficult. As a result, we implemented stores and [hooks] to solve this problem.

references

#938
#921
#874

v1.148.0

Colorize `atmos describe` commands output when TTY is attached @Cerebrovinny (#919)

what

• Colorize atmos describe commands output when TTY is attached
• Added syntax highlighting configuration for terminal outputs
• Introduced customizable settings for terminal display, including line numbers and color styles
• Enhanced configuration options for YAML and JSON output formatting

why

• Improve UX

test

Before:

After:

v1.147.0

Validate Atmos Log Levels @Cerebrovinny (#930)

what

• Validate Atmos Log Levels
• Error if invalid log level is set via the ATMOS_LOGS_LEVEL ENV variable or in atmos.yaml

why

• Don't accept invalid log levels