[aws-cloudfront & aws-cloudfront-origins] Adding an existing bucket as an S3Origin

[robertd]

❓ General Issue

Hello, @njlynch. I've been trying to use the new cloudformation.Distribution L2 constructor. I'm trying to replicate what aws-solutions-constructs is did with their L3 constructs while waiting for their official support/rewrite.

I've noticed that the new S3Origin construct in aws-cloudfront-origins creates OriginAccessIdentity under the hood for the cloudfront distro, but that OAI is not accessible (unless using hatch escape technique) to apply it to the existing (or brand new) s3 bucket through bucket policy. Should the new Distribution auto-magically try to update bucket policy and try to apply OAI to the provided s3 bucket? Is this in the works by any chance?

Or... perhaps if OAI should be exposed through S3Origin or Distribution constructs?

import * as s3 from '@aws-cdk/aws-s3';
import * as cloudfront from '@aws-cdk/aws-cloudfront';
import * as origins from '@aws-cdk/aws-cloudfront-origins';

...

    const s3LogBucket = s3.Bucket.fromBucketName(this, "logs-bucket", `log-bucket-${props.env.account}`);
    const contentBucket = s3.Bucket.fromBucketName(this, "static-bucket", `static-content-${props.env.account}`);

    const certificate = acm.Certificate.fromCertificateArn(
      this, 
      "certificate", 
      `arn:aws:acm:us-east-2:${props.env.account}:certificate/40cdd40c-a3f4-4131-9643-1234567890`
    );

    const distro = new cloudfront.Distribution(this, "cf-distro", {
      defaultBehavior: {
        origin: new origins.S3Origin(contentBucket),
        allowedMethods: cloudfront.AllowedMethods.ALLOW_GET_HEAD_OPTIONS,
        viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
      },
      certificate,
      logBucket: s3LogBucket,
      logFilePrefix: `AWSLogs/${props.env.account}/cloudfront/`,
      defaultRootObject: "index.html",
      priceClass: cloudfront.PriceClass.PRICE_CLASS_100
    });

Error I'm getting when trying to access the CF distro.

Environment

• CDK CLI Version: 1.59.0 (build 1d082f4)
• Module Version: 1.59.0 (build 1d082f4)
• Node.js Version: v14.8.0
• OS: OSX Mojave
• Language (Version): TypeScript (3.8.3)

Other information

[robertd]

@njlynch Upon further experimenting... I've noticed that if I'm creating new bucket from the scratch, CloudFront distribution will update bucket policies with newly created OAI information.

staticcontentbucketPolicyE0BDBE43:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: staticcontentbucket64C926A2
      PolicyDocument:
        Statement:
          - Action:
              - s3:GetObject*
              - s3:GetBucket*
              - s3:List*
            Effect: Allow
            Principal:
              CanonicalUser:
                Fn::GetAtt:
                  - cfdistroOrigin1S3OriginC04A5F28
                  - S3CanonicalUserId
            Resource:
              - Fn::GetAtt:
                  - staticcontentbucket64C926A2
                  - Arn
              - Fn::Join:
                  - ""
                  - - Fn::GetAtt:
                        - staticcontentbucket64C926A2
                        - Arn
                    - /*
        Version: "2012-10-17"

However, If I try and use existing bucket, no bucket policies are being added.

[njlynch]

Upon further experimenting... I've noticed that if I'm creating new bucket from the scratch, CloudFront distribution will update bucket policies with newly created OAI information. However, If I try and use existing bucket, no bucket policies are being added.

This is general behavior in the CDK for imported buckets -- and most other imported resources; in this case, since the bucket is imported, we can't know if there is an existing bucket policy or not and whether a new policy can (or should) be created, so the grant* methods are no-ops.

Note that the OAI and bucket policy are only necessary if the bucket policy/permissions otherwise restrict access to the bucket such that CloudFront wouldn't be able to access the bucket without them. If the bucket is public (for example), then the OAI is redundant.

Depending on how the other buckets are defined, you could either pass the buckets in (rather than importing them), use escape hatches (for now) to get at the OAI and explicitly create a bucket policy (if you know that no other bucket policies do or will exist on the buckets), or if you don't need the OAI, leave as-is. I'd certainly be open to a feature request (and PR!) to expose the OAI to avoid the need for escape hatches.

[robertd]

@njlynch thanks for the detailed response. OAI feature would a great addition . :)

[njlynch]

Sure. I've opened #9859 to track the feature request. Pull requests welcome! Closing this issue out in favor of the feature request.

[wchaws]

Try this:

import { App, Stack, StackProps } from '@aws-cdk/core';
import { CloudFrontWebDistribution, OriginAccessIdentity } from '@aws-cdk/aws-cloudfront';
import { Bucket, BucketPolicy } from '@aws-cdk/aws-s3';
import { PolicyStatement } from '@aws-cdk/aws-iam';

export class CloudfrontS3Stack extends Stack {
  constructor(scope: App, id: string, props?: StackProps) {
    super(scope, id, props);

    const testBucket = Bucket.fromBucketName(this, 'TestBucket', 'dmahapatro-personal-bucket');

    // Create Origin Access Identity to be use Canonical User Id in S3 bucket policy
    const originAccessIdentity = new OriginAccessIdentity(this, 'OAI', {
      comment: "Created_by_dmahapatro"
    });

    // This does not seem to work if Bucket.fromBucketName is used
    // It works for S3 buckets which are created as part of this stack
    // testBucket.grantRead(originAccessIdentity);

    // Explicitly add Bucket Policy 
    const policyStatement = new PolicyStatement();
    policyStatement.addActions('s3:GetBucket*');
    policyStatement.addActions('s3:GetObject*');
    policyStatement.addActions('s3:List*');
    policyStatement.addResources(testBucket.bucketArn);
    policyStatement.addResources(`${testBucket.bucketArn}/*`);
    policyStatement.addCanonicalUserPrincipal(originAccessIdentity.cloudFrontOriginAccessIdentityS3CanonicalUserId);

    // testBucket.addToResourcePolicy(policyStatement);

    // Manually create or update bucket policy
    if( !testBucket.policy ) {
      new BucketPolicy(this, 'Policy', { bucket: testBucket }).document.addStatements(policyStatement);
    } else {
      testBucket.policy.document.addStatements(policyStatement);
    }

    // Create Cloudfront distribution with S3 as Origin
    const distribution = new CloudFrontWebDistribution(this, 'cdk-example-distribution', {
      originConfigs: [
        {
          s3OriginSource: {
            s3BucketSource: testBucket,
            originAccessIdentity: originAccessIdentity
          },
          behaviors: [
            { isDefaultBehavior: true }
          ]
        }
      ]
    });
  }
}

from: https://stackoverflow.com/a/60917015/4108187