Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bootstrap): customizable bootstrap template #9886

Merged
merged 6 commits into from
Aug 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ import * as sinon from 'sinon';
import { AwsSdkCall, PhysicalResourceId } from '../../lib';
import { flatten, handler, forceSdkInstallation } from '../../lib/aws-custom-resource/runtime';


// This test performs an 'npm install' which may take longer than the default
// 5s timeout
jest.setTimeout(60_000);

/* eslint-disable no-console */

console.log = jest.fn();
Expand Down
10 changes: 2 additions & 8 deletions packages/aws-cdk/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,10 @@ $ test/integ/run-against-dist test/integ/cli/test.sh
$ test/integ/run-against-release test/integ/cli/test.sh
```

You can switch out the test script to run the init tests:
To run a single integ test in the source tree:

```
$ test/integ/run-against-xxx test/integ/init/test-all.sh
```

Or even run a single integ test:

```
$ test/integ/run-against-xxx test/integ/init/test-cdk-deploy-no-tty.sh
$ test/integ/run-against-repo test/integ/cli/test.sh -t 'SUBSTRING OF THE TEST NAME'
```

### CLI integration tests
Expand Down
18 changes: 16 additions & 2 deletions packages/aws-cdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ Example `outputs.json` after deployment of multiple stacks
},
"AnotherStack": {
"VPCId": "vpc-z0mg270fee16693f"
}
}
}
```

Expand Down Expand Up @@ -277,9 +277,23 @@ $ # Deploys only to environments foo and bar
$ cdk bootstrap --app='node bin/main.js' foo bar
```

By default, bootstrap stack will be protected from stack termination. This can be disabled using
By default, bootstrap stack will be protected from stack termination. This can be disabled using
`--termination-protection` argument.

If you have specific needs, policies, or requirements not met by the default template, you can customize it
to fit your own situation, by exporting the default one to a file and either deploying it yourself
using CloudFormation directly, or by telling the CLI to use a custom template. That looks as follows:

```console
# Dump the built-in template to a file
$ cdk bootstrap --show-template > bootstrap-template.yaml

# Edit 'bootstrap-template.yaml' to your liking

# Tell CDK to use the customized template
$ cdk bootstrap --template bootstrap-template.yaml
```

#### `cdk doctor`
Inspect the current command-line environment and configurations, and collect information that can be useful for
troubleshooting problems. It is usually a good idea to include the information provided by this command when submitting
Expand Down
46 changes: 31 additions & 15 deletions packages/aws-cdk/bin/cdk.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import * as cxapi from '@aws-cdk/cx-api';
import * as colors from 'colors/safe';
import * as yargs from 'yargs';

import { ToolkitInfo } from '../lib';
import { ToolkitInfo, BootstrapSource, Bootstrapper } from '../lib';
import { SdkProvider } from '../lib/api/aws-auth';
import { CloudFormationDeployments } from '../lib/api/cloudformation-deployments';
import { CloudExecutable } from '../lib/api/cxapp/cloud-executable';
Expand Down Expand Up @@ -77,7 +77,9 @@ async function parseCommandLineArguments() {
.option('trust', { type: 'array', desc: 'The AWS account IDs that should be trusted to perform deployments into this environment (may be repeated)', default: [], nargs: 1, requiresArg: true, hidden: true })
.option('cloudformation-execution-policies', { type: 'array', desc: 'The Managed Policy ARNs that should be attached to the role performing deployments into this environment. Required if --trust was passed (may be repeated)', default: [], nargs: 1, requiresArg: true, hidden: true })
.option('force', { alias: 'f', type: 'boolean', desc: 'Always bootstrap even if it would downgrade template version', default: false })
.option('termination-protection', { type: 'boolean', default: false, desc: 'Toggle CloudFormation termination protection on the bootstrap stacks' }),
.option('termination-protection', { type: 'boolean', default: false, desc: 'Toggle CloudFormation termination protection on the bootstrap stacks' })
.option('show-template', { type: 'boolean', desc: 'Instead of actual bootstrapping, print the current CLI\'s bootstrapping template to stdout for customization.', default: false })
.option('template', { type: 'string', requiresArg: true, desc: 'Use the template from the given file instead of the built-in one (use --show-template to obtain an example).' }),
)
.command('deploy [STACKS..]', 'Deploys the stack(s) named STACKS into your AWS account', yargs => yargs
.option('build-exclude', { type: 'array', alias: 'E', nargs: 1, desc: 'Do not rebuild asset with the given ID. Can be specified multiple times.', default: [] })
Expand Down Expand Up @@ -234,30 +236,44 @@ async function initCommandLine() {
case 'bootstrap':
// Use new bootstrapping if it's requested via environment variable, or if
// new style stack synthesis has been configured in `cdk.json`.
let useNewBootstrapping = false;
if (process.env.CDK_NEW_BOOTSTRAP) {
//
// In code it's optimistically called "default" bootstrapping but that is in
njlynch marked this conversation as resolved.
Show resolved Hide resolved
// anticipation of flipping the switch, in user messaging we still call it
// "new" bootstrapping.
let source: BootstrapSource = { source: 'legacy' };
if (args.template) {
print(`Using bootstrapping template from ${args.template}`);
source = { source: 'custom', templateFile: args.template };
} else if (process.env.CDK_NEW_BOOTSTRAP) {
print('CDK_NEW_BOOTSTRAP set, using new-style bootstrapping');
useNewBootstrapping = true;
source = { source: 'default' };
} else if (configuration.context.get(cxapi.NEW_STYLE_STACK_SYNTHESIS_CONTEXT)) {
print(`'${cxapi.NEW_STYLE_STACK_SYNTHESIS_CONTEXT}' context set, using new-style bootstrapping`);
useNewBootstrapping = true;
source = { source: 'default' };
}

return await cli.bootstrap(args.ENVIRONMENTS, toolkitStackName,
args.roleArn,
useNewBootstrapping,
argv.force,
{
const bootstrapper = new Bootstrapper(source);

if (args.showTemplate) {
return await bootstrapper.showTemplate();
}

return await cli.bootstrap(args.ENVIRONMENTS, bootstrapper, {
roleArn: args.roleArn,
force: argv.force,
toolkitStackName: toolkitStackName,
execute: args.execute,
tags: configuration.settings.get(['tags']),
terminationProtection: args.terminationProtection,
parameters: {
bucketName: configuration.settings.get(['toolkitBucket', 'bucketName']),
kmsKeyId: configuration.settings.get(['toolkitBucket', 'kmsKeyId']),
qualifier: args.qualifier,
publicAccessBlockConfiguration: args.publicAccessBlockConfiguration,
tags: configuration.settings.get(['tags']),
execute: args.execute,
trustedAccounts: args.trust,
cloudFormationExecutionPolicies: args.cloudformationExecutionPolicies,
terminationProtection: args.terminationProtection,
});
},
});

case 'deploy':
const parameterMap: { [name: string]: string | undefined } = {};
Expand Down
161 changes: 107 additions & 54 deletions packages/aws-cdk/lib/api/bootstrap/bootstrap-environment.ts
Original file line number Diff line number Diff line change
@@ -1,70 +1,123 @@
import * as path from 'path';
import * as cxapi from '@aws-cdk/cx-api';
import { loadStructuredFile } from '../../serialize';
import { loadStructuredFile, toYAML } from '../../serialize';
import { SdkProvider } from '../aws-auth';
import { DeployStackResult } from '../deploy-stack';
import { BootstrapEnvironmentOptions } from './bootstrap-props';
import { deployBootstrapStack } from './deploy-bootstrap';
import { BootstrapEnvironmentOptions, BootstrappingParameters } from './bootstrap-props';
import { deployBootstrapStack, bootstrapVersionFromTemplate } from './deploy-bootstrap';
import { legacyBootstrapTemplate } from './legacy-template';

/* eslint-disable max-len */

/**
* Deploy legacy bootstrap stack
*
* @experimental
*/
export async function bootstrapEnvironment(environment: cxapi.Environment, sdkProvider: SdkProvider, options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {
const params = options.parameters ?? {};
export type BootstrapSource =
{ source: 'legacy' }
| { source: 'default' }
| { source: 'custom'; templateFile: string };

if (params.trustedAccounts?.length) {
throw new Error('--trust can only be passed for the new bootstrap experience.');

export class Bootstrapper {
constructor(private readonly source: BootstrapSource) {
}

public bootstrapEnvironment(environment: cxapi.Environment, sdkProvider: SdkProvider, options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {
switch (this.source.source) {
case 'legacy':
return this.legacyBootstrap(environment, sdkProvider, options);
case 'default':
return this.defaultBootstrap(environment, sdkProvider, options);
case 'custom':
return this.customBootstrap(environment, sdkProvider, options);
}
}

public async showTemplate() {
const template = await this.loadTemplate();
process.stdout.write(`${toYAML(template)}\n`);
}

/**
* Deploy legacy bootstrap stack
*
* @experimental
*/
private async legacyBootstrap(environment: cxapi.Environment, sdkProvider: SdkProvider, options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {
const params = options.parameters ?? {};

if (params.trustedAccounts?.length) {
throw new Error('--trust can only be passed for the new bootstrap experience.');
}
if (params.cloudFormationExecutionPolicies?.length) {
throw new Error('--cloudformation-execution-policies can only be passed for the new bootstrap experience.');
}
if (params.qualifier) {
throw new Error('--qualifier can only be passed for the new bootstrap experience.');
}

return deployBootstrapStack(
await this.loadTemplate(params),
{},
environment,
sdkProvider,
options);
}
if (params.cloudFormationExecutionPolicies?.length) {
throw new Error('--cloudformation-execution-policies can only be passed for the new bootstrap experience.');

/**
* Deploy CI/CD-ready bootstrap stack from template
*
* @experimental
*/
private async defaultBootstrap(
environment: cxapi.Environment,
sdkProvider: SdkProvider,
options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {

const params = options.parameters ?? {};

if (params.trustedAccounts?.length && !params.cloudFormationExecutionPolicies?.length) {
throw new Error('--cloudformation-execution-policies are required if --trust has been passed!');
}

const bootstrapTemplate = await this.loadTemplate();

return deployBootstrapStack(
bootstrapTemplate,
{
FileAssetsBucketName: params.bucketName,
FileAssetsBucketKmsKeyId: params.kmsKeyId,
TrustedAccounts: params.trustedAccounts?.join(','),
CloudFormationExecutionPolicies: params.cloudFormationExecutionPolicies?.join(','),
Qualifier: params.qualifier,
PublicAccessBlockConfiguration: params.publicAccessBlockConfiguration || params.publicAccessBlockConfiguration === undefined ? 'true' : 'false',
},
environment,
sdkProvider,
options);
}
if (params.qualifier) {
throw new Error('--qualifier can only be passed for the new bootstrap experience.');

private async customBootstrap(
environment: cxapi.Environment,
sdkProvider: SdkProvider,
options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {

// Look at the template, decide whether it's most likely a legacy or modern bootstrap
// template, and use the right bootstrapper for that.
const version = bootstrapVersionFromTemplate(await this.loadTemplate());
if (version === 0) {
return this.legacyBootstrap(environment, sdkProvider, options);
} else {
return this.defaultBootstrap(environment, sdkProvider, options);
}
}

return deployBootstrapStack(
legacyBootstrapTemplate(params),
{},
environment,
sdkProvider,
options);
}

/**
* Deploy CI/CD-ready bootstrap stack from template
*
* @experimental
*/
export async function bootstrapEnvironment2(
environment: cxapi.Environment,
sdkProvider: SdkProvider,
options: BootstrapEnvironmentOptions = {}): Promise<DeployStackResult> {

const params = options.parameters ?? {};

if (params.trustedAccounts?.length && !params.cloudFormationExecutionPolicies?.length) {
throw new Error('--cloudformation-execution-policies are required if --trust has been passed!');
private async loadTemplate(params: BootstrappingParameters = {}): Promise<any> {
njlynch marked this conversation as resolved.
Show resolved Hide resolved
switch (this.source.source) {
case 'custom':
return loadStructuredFile(this.source.templateFile);
case 'default':
return loadStructuredFile(path.join(__dirname, 'bootstrap-template.yaml'));
case 'legacy':
return legacyBootstrapTemplate(params);
}
}

const bootstrapTemplatePath = path.join(__dirname, 'bootstrap-template.yaml');
const bootstrapTemplate = await loadStructuredFile(bootstrapTemplatePath);

return deployBootstrapStack(
bootstrapTemplate,
{
FileAssetsBucketName: params.bucketName,
FileAssetsBucketKmsKeyId: params.kmsKeyId,
TrustedAccounts: params.trustedAccounts?.join(','),
CloudFormationExecutionPolicies: params.cloudFormationExecutionPolicies?.join(','),
Qualifier: params.qualifier,
PublicAccessBlockConfiguration: params.publicAccessBlockConfiguration || params.publicAccessBlockConfiguration === undefined ? 'true' : 'false',
},
environment,
sdkProvider,
options);
}
37 changes: 20 additions & 17 deletions packages/aws-cdk/lib/api/bootstrap/bootstrap-props.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,26 @@ export interface BootstrapEnvironmentOptions {
readonly roleArn?: string;
readonly parameters?: BootstrappingParameters;
readonly force?: boolean;

/**
* Whether to execute the changeset or only create it and leave it in review.
* @default true
*/
readonly execute?: boolean;

/**
* Tags for cdktoolkit stack.
*
* @default - None.
*/
readonly tags?: Tag[];

/**
* Whether the stacks created by the bootstrap process should be protected from termination.
* @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-protect-stacks.html
* @default true
*/
readonly terminationProtection?: boolean;
}

/**
Expand All @@ -38,17 +58,6 @@ export interface BootstrappingParameters {
* @default - the default KMS key for S3 will be used.
*/
readonly kmsKeyId?: string;
/**
* Tags for cdktoolkit stack.
*
* @default - None.
*/
readonly tags?: Tag[];
/**
* Whether to execute the changeset or only create it and leave it in review.
* @default true
*/
readonly execute?: boolean;

/**
* The list of AWS account IDs that are trusted to deploy into the environment being bootstrapped.
Expand Down Expand Up @@ -80,10 +89,4 @@ export interface BootstrappingParameters {
*/
readonly publicAccessBlockConfiguration?: boolean;

/**
* Whether the stacks created by the bootstrap process should be protected from termination.
* @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-protect-stacks.html
* @default true
*/
readonly terminationProtection?: boolean;
}
Loading