Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support Component Lock with metadata.locked #908

Merged
merged 34 commits into from
Jan 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
7dae7f9
Add support for locking components with metadata.locked
milldr Jan 3, 2025
08bc176
Update function call to include an additional parameter
milldr Jan 3, 2025
bdc95a3
Adding example for testing
milldr Jan 3, 2025
2938bdc
Add test cases for atmos terraform deploy commands
milldr Jan 3, 2025
6278761
Remove unnecessary stdout and stderr checks
milldr Jan 3, 2025
01d5db9
move examples/tests to tests/fixtures/complete
milldr Jan 6, 2025
4baa894
replace all s|examples/tests|tests/fixtures/scenarios/complete|g
milldr Jan 6, 2025
88637c6
replace all s|examples/tests|tests/fixtures/scenarios/complete|g
milldr Jan 6, 2025
6ecb5fe
use common component directory for test fixtures
milldr Jan 6, 2025
1361f75
debugging test example rename
milldr Jan 6, 2025
a3de909
Merge branch 'main' into chore/DEV-2906_test-fixtures
milldr Jan 6, 2025
b662a49
Update demo-folder paths and fix working-directory in test.yml
milldr Jan 6, 2025
9e09744
removed shared components
milldr Jan 6, 2025
6451aee
Remove cache.yaml from demo-stacks directory
milldr Jan 6, 2025
a79ba8d
merged main
milldr Jan 6, 2025
2519157
Merge branch 'chore/DEV-2906_test-fixtures' into feat/DEV-2849_metada…
milldr Jan 6, 2025
c4c94d1
create test fixture for metadata
milldr Jan 6, 2025
4c9087f
Merge branch 'main' into feat/DEV-2849_metadata-locked
milldr Jan 6, 2025
1355349
Update test case to include expected exit code 1
milldr Jan 6, 2025
682cdfd
Add atmos schema with locked flag in atmos manifest
milldr Jan 6, 2025
9a994be
support separate test case files, added metadata tests
milldr Jan 6, 2025
eb3ab97
support separate test case files, added metadata tests
milldr Jan 6, 2025
4965f7c
Remove 'secrets' option from locked component commands
milldr Jan 7, 2025
880240e
Merge branch 'main' into feat/DEV-2849_metadata-locked
aknysh Jan 7, 2025
1c669f1
Add 'locked' property in atmos-manifest schema
milldr Jan 7, 2025
d85e9cd
Add ignore rule for cache text files
milldr Jan 7, 2025
47720b2
Merge branch 'main' into feat/DEV-2849_metadata-locked
milldr Jan 8, 2025
db0eace
Merge branch 'main' into feat/DEV-2849_metadata-locked
milldr Jan 9, 2025
225364e
Merge branch 'feat/DEV-2849_metadata-locked' of github.com:cloudposse…
milldr Jan 9, 2025
1e6f375
removed schema
milldr Jan 9, 2025
8aaf679
removed schema
milldr Jan 9, 2025
a70564c
Merge branch 'main' into feat/DEV-2849_metadata-locked
milldr Jan 10, 2025
43f3cb5
clean up doc
milldr Jan 10, 2025
3dbfc55
Merge branch 'main' of github.com:cloudposse/atmos into feat/DEV-2849…
milldr Jan 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions internal/exec/helmfile.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,16 @@ func ExecuteHelmfile(info schema.ConfigAndStacksInfo) error {
"by 'metadata.type: abstract' attribute", filepath.Join(info.ComponentFolderPrefix, info.Component))
}

// Check if the component is locked (`metadata.locked` is set to true)
if info.ComponentIsLocked {
// Allow read-only commands, block modification commands
switch info.SubCommand {
case "sync", "apply", "deploy", "delete", "destroy":
return fmt.Errorf("component '%s' is locked and cannot be modified (metadata.locked = true)",
filepath.Join(info.ComponentFolderPrefix, info.Component))
}
}

// Print component variables
u.LogDebug(atmosConfig, fmt.Sprintf("\nVariables for the component '%s' in the stack '%s':", info.ComponentFromArg, info.Stack))

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,10 @@
"description": "Custom configuration per component, not inherited by derived components",
"additionalProperties": true,
"title": "custom"
},
"locked": {
"type": "boolean",
"description": "Flag to lock the component and prevent modifications while allowing read operations"
}
},
"required": [],
Expand Down
12 changes: 9 additions & 3 deletions internal/exec/stack_utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,15 @@ func BuildTerraformWorkspace(atmosConfig schema.AtmosConfiguration, configAndSta
}

// ProcessComponentMetadata processes component metadata and returns a base component (if any) and whether
// the component is real or abstract and whether the component is disabled or not
// the component is real or abstract and whether the component is disabled or not and whether the component is locked
func ProcessComponentMetadata(
component string,
componentSection map[string]any,
) (map[string]any, string, bool, bool) {
) (map[string]any, string, bool, bool, bool) {
baseComponentName := ""
componentIsAbstract := false
componentIsEnabled := true
componentIsLocked := false
var componentMetadata map[string]any

// Find base component in the `component` attribute
Expand All @@ -82,6 +83,11 @@ func ProcessComponentMetadata(
componentIsEnabled = false
}
}
if lockedValue, exists := componentMetadata["locked"]; exists {
if locked, ok := lockedValue.(bool); ok && locked {
componentIsLocked = true
}
}
// Find base component in the `metadata.component` attribute
// `metadata.component` overrides `component`
if componentMetadataComponent, componentMetadataComponentExists := componentMetadata[cfg.ComponentSectionName].(string); componentMetadataComponentExists {
Expand All @@ -94,7 +100,7 @@ func ProcessComponentMetadata(
baseComponentName = ""
}

return componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled
return componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled, componentIsLocked
}

// BuildDependentStackNameFromDependsOnLegacy builds the dependent stack name from "settings.spacelift.depends_on" config
Expand Down
10 changes: 10 additions & 0 deletions internal/exec/terraform.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,16 @@ func ExecuteTerraform(info schema.ConfigAndStacksInfo) error {
"by 'metadata.type: abstract' attribute", filepath.Join(info.ComponentFolderPrefix, info.Component))
}

// Check if the component is locked (`metadata.locked` is set to true)
if info.ComponentIsLocked {
// Allow read-only commands, block modification commands
switch info.SubCommand {
case "apply", "deploy", "destroy", "import", "state", "taint", "untaint":
return fmt.Errorf("component '%s' is locked and cannot be modified (metadata.locked = true)",
filepath.Join(info.ComponentFolderPrefix, info.Component))
}
}

if info.SubCommand == "clean" {
err := handleCleanSubCommand(info, componentPath, atmosConfig)
if err != nil {
Expand Down
6 changes: 4 additions & 2 deletions internal/exec/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,9 @@ func ProcessComponentConfig(
}

// Process component metadata and find a base component (if any) and whether the component is real or abstract
componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled := ProcessComponentMetadata(component, componentSection)
componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled, componentIsLocked := ProcessComponentMetadata(component, componentSection)
configAndStacksInfo.ComponentIsEnabled = componentIsEnabled
configAndStacksInfo.ComponentIsLocked = componentIsLocked

// Remove the ENV vars that are set to `null` in the `env` section.
// Setting an ENV var to `null` in stack config has the effect of unsetting it
Expand Down Expand Up @@ -610,9 +611,10 @@ func ProcessStacks(
configAndStacksInfo.ComponentEnvList = u.ConvertEnvVars(configAndStacksInfo.ComponentEnvSection)

// Process component metadata
_, baseComponentName, _, componentIsEnabled := ProcessComponentMetadata(configAndStacksInfo.ComponentFromArg, configAndStacksInfo.ComponentSection)
_, baseComponentName, _, componentIsEnabled, componentIsLocked := ProcessComponentMetadata(configAndStacksInfo.ComponentFromArg, configAndStacksInfo.ComponentSection)
configAndStacksInfo.BaseComponentPath = baseComponentName
configAndStacksInfo.ComponentIsEnabled = componentIsEnabled
configAndStacksInfo.ComponentIsLocked = componentIsLocked

// Process component path and name
configAndStacksInfo.ComponentFolderPrefix = ""
Expand Down
1 change: 1 addition & 0 deletions pkg/schema/schema.go
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ type ConfigAndStacksInfo struct {
NeedHelp bool
ComponentIsAbstract bool
ComponentIsEnabled bool
ComponentIsLocked bool
ComponentMetadataSection AtmosSectionMapType
TerraformWorkspace string
JsonSchemaDir string
Expand Down
2 changes: 1 addition & 1 deletion pkg/spacelift/spacelift_stack_processor.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ func TransformStackConfigToSpaceliftStacks(
}

// Process component metadata and find a base component (if any) and whether the component is real or abstract
componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled := e.ProcessComponentMetadata(component, componentMap)
componentMetadata, baseComponentName, componentIsAbstract, componentIsEnabled, _ := e.ProcessComponentMetadata(component, componentMap)

if componentIsAbstract || !componentIsEnabled {
continue
Expand Down
50 changes: 50 additions & 0 deletions tests/fixtures/components/terraform/myapp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Example Terraform Weather Component

This Terraform "root" module fetches weather information for a specified location with custom display options.
It queries data from the [`wttr.in`](https://wttr.in) weather service and stores the result in a local file (`cache.txt`).
It also provides several outputs like weather information, request URL, stage, location, language, and units of measurement.

## Features

- Fetch weather updates for a location using HTTP request.
- Write the obtained weather data in a local file.
- Customizable display options.
- View the request URL.
- Get informed about the stage, location, language, and units in the metadata.

## Usage

To include this module in your [Atmos Stacks](https://atmos.tools/core-concepts/stacks) configuration:

```yaml
components:
terraform:
weather:
vars:
stage: dev
location: New York
options: 0T
format: v2
lang: en
units: m
```

### Inputs
- `stage`: Stage where it will be deployed.
- `location`: Location for which the weather is reported. Default is "Los Angeles".
- `options`: Options to customize the output. Default is "0T".
- `format`: Specifies the output format. Default is "v2".
- `lang`: Language in which the weather will be displayed. Default is "en".
- `units`: Units in which the weather will be displayed. Default is "m".

### Outputs
- `weather`: The fetched weather data.
- `url`: Requested URL.
- `stage`: Stage of deployment.
- `location`: Location of the reported weather.
- `lang`: Language used for weather data.
- `units`: Units of measurement for the weather data.

Please note, this module requires Terraform version >=1.0.0, and you need to specify no other required providers.

Happy Weather Tracking!
22 changes: 22 additions & 0 deletions tests/fixtures/components/terraform/myapp/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
locals {
url = format("https://wttr.in/%v?%v&format=%v&lang=%v&u=%v",
urlencode(var.location),
urlencode(var.options),
urlencode(var.format),
urlencode(var.lang),
urlencode(var.units),
)
}

data "http" "weather" {
url = local.url
request_headers = {
User-Agent = "curl"
}
}
milldr marked this conversation as resolved.
Show resolved Hide resolved

# Now write this to a file (as an example of a resource)
resource "local_file" "cache" {
filename = "cache.${var.stage}.txt"
content = data.http.weather.response_body
}
milldr marked this conversation as resolved.
Show resolved Hide resolved
27 changes: 27 additions & 0 deletions tests/fixtures/components/terraform/myapp/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
output "weather" {
value = data.http.weather.response_body
}

output "url" {
value = local.url
}

output "stage" {
value = var.stage
description = "Stage where it was deployed"
}

output "location" {
value = var.location
description = "Location of the weather report."
}

output "lang" {
value = var.lang
description = "Language which the weather is displayed."
}

output "units" {
value = var.units
description = "Units the weather is displayed."
}
34 changes: 34 additions & 0 deletions tests/fixtures/components/terraform/myapp/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
variable "stage" {
description = "Stage where it will be deployed"
type = string
}

variable "location" {
description = "Location for which the weather."
type = string
default = "Los Angeles"
}

variable "options" {
description = "Options to customize the output."
type = string
default = "0T"
}

variable "format" {
description = "Format of the output."
type = string
default = "v2"
}

variable "lang" {
description = "Language in which the weather is displayed."
type = string
default = "en"
}

variable "units" {
description = "Units in which the weather is displayed."
type = string
default = "m"
}
5 changes: 5 additions & 0 deletions tests/fixtures/components/terraform/myapp/versions.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
terraform {
required_version = ">= 1.0.0"

required_providers {}
}
10 changes: 10 additions & 0 deletions tests/fixtures/scenarios/metadata/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
**/.terraform/**
**/.terraform.lock.hcl
**/.terraform.tfstate/**
**/.terraform.tfstate.backup/**
**/*.tfvars.json
**/*.planfile
**/terraform.tfstate
**/terraform.tfstate.backup
**/terraform.tfstate.d/**
**/cache.*.txt
21 changes: 21 additions & 0 deletions tests/fixtures/scenarios/metadata/atmos.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
base_path: "./"

components:
terraform:
base_path: "../../components/terraform"
apply_auto_approve: false
deploy_run_init: true
init_run_reconfigure: true
auto_generate_backend_file: false

stacks:
base_path: "stacks"
included_paths:
- "deploy/**/*"
excluded_paths:
- "**/_defaults.yaml"
name_pattern: "{stage}"

logs:
file: "/dev/stderr"
level: Info
11 changes: 11 additions & 0 deletions tests/fixtures/scenarios/metadata/stacks/catalog/myapp.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# yaml-language-server: $schema=https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json

components:
terraform:
myapp:
vars:
location: Los Angeles
lang: en
format: ''
options: '0'
units: m
14 changes: 14 additions & 0 deletions tests/fixtures/scenarios/metadata/stacks/deploy/nonprod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# yaml-language-server: $schema=https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json

vars:
stage: nonprod

import:
- catalog/myapp

components:
terraform:
myapp:
vars:
location: Stockholm
lang: se
16 changes: 16 additions & 0 deletions tests/fixtures/scenarios/metadata/stacks/deploy/prod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# yaml-language-server: $schema=https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json

vars:
stage: prod

import:
- catalog/myapp

components:
terraform:
myapp:
metadata:
locked: true
vars:
location: Los Angeles
lang: en
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,10 @@
"description": "Custom configuration per component, not inherited by derived components",
"additionalProperties": true,
"title": "custom"
},
milldr marked this conversation as resolved.
Show resolved Hide resolved
"locked": {
"type": "boolean",
"description": "Flag to lock the component and prevent modifications while allowing read operations"
}
},
"required": [],
Expand Down
Loading
Loading