Self-updating module considerations

[marshall7m]

If a new version of this module was applied to a user's AWS account within it's own Step Function execution, the downstream executions may be effected.

For example, a version of this module may introduce new metadb columns to the executions or account_dim tables. The trigger SF lambda function now handles these new columns but the running execution records will not have valid values for these new columns. This will cause the Trigger SF lambda function to break since it will requesting the new columns from the old record.

Potential solutions:

• Refactor create_metadb_tables.sql and create_metadb_users.sql to update attributes if table/user exists rather than skipping the query operation entirely when running the terraform apply on the module

• To propagate the new columns to the existing running execution records, rerun the Create Deploy Stack ECS task with the same parameters. Refactor the create deploy stack SQL to also update the records if they exists.

• Put a hard limitation on this module saying that there should be no downstream dependencies on this module. This will prevent any inconsistency within metadb records and CI pipeline versioning within the scope of the PR. This will ensure that the new changes will only be applied to future PRs. Does bring the tradeoff of users not being able to define Terragrunt dependency blocks using this module.