[apigateway] Cyclic dependency between APIGateway and SQS

[praman28]

I created an APIGateway in one stack, and SQS and the respective APIGateway Resource in front of SQS in a separate stack. On running the cdk synth the code results with a cyclic dependency error as below:

Error: 'data-event-processor-stack' depends on 'dw-api' (data-event-processor-stack/queue-api/Resource -> dw-api/DWAPIGateway/Resource.RootResourceId). Adding this dependency (dependency added using stack.addDependency()) would create a cyclic reference.

As per the issue ##4010 such issue is already fixed in v1.9.0. However we are still facing either the same or similar issue.

Reproduction Steps

Below is the code snippet that can be used to recreate the scenario.

SQS Queue stack

export class DataEventProcessorStack extends cdk.Stack {

    private eventSubscriberQueue: IQueue;
    private readonly eventQueueName: string = 'dw-data-event-queue';

    constructor(scope: cdk.Construct, id: string, private readonly props: DataEventProcessorStackProps) {
        super(scope, id, props);
        this.createQueue();
        this.registerQueueAsApi();
    }

    private createQueue() {
        this.eventSubscriberQueue = new Queue(this, `DataEventProcessorQueue`, {
            queueName : `${this.props.audience}-${this.eventQueueName}`,
            retentionPeriod: Duration.days(14),
            visibilityTimeout: Duration.minutes(5),
        });
        this.eventSubscriberQueue.grant({grantPrincipal : new AnyPrincipal()}, 'sqs:SendMessage');
        this.lambdaFunction.addEventSource(new SqsEventSource(this.eventSubscriberQueue));
    }

    private registerQueueAsApi() {

        const queueIntegration = new ApiGateway.AwsIntegration({
            service: 'SQS',
            integrationHttpMethod: 'POST',
            path: this.eventSubscriberQueue.queueName,
        });

        const resource = new ApiGateway.Resource(this, 'queue-api', {
            parent: this.props.apiGateway.root,
            pathPart: 'queue',
        });
        resource.addMethod('POST', queueIntegration);

    }

}

APIGateway stack

export class APIStack extends cdk.Stack {

    private apiGateway: ApiGateway.RestApi;

    constructor(scope: cdk.Construct, id: string, private readonly props: APIStackProps) {
        super(scope, id, props);

        this.createApiGateway();
    }

    getApiGateway(): ApiGateway.RestApi {
        return this.apiGateway;
    }

    createApiGateway() {
        this.apiGateway = new ApiGateway.RestApi(this, 'DWAPIGateway', {
            description: 'Data warehouse API gateway',
            restApiName: `${this.props.audience}-dw`,
            deployOptions: {
                stageName: 'prod',
                tracingEnabled: true,
            },
        });
        this.apiGateway.root.addMethod('ANY');
    }
}

Index.ts

export interface StackProps extends cdk.StackProps {
    audience: string;
    logLevel: string;
}

const app = new cdk.App();
const config = new Config(process.env);
const props: StackProps = {
    audience: config.getAudience(),
    env: {
        account: config.getAWSAccount(),
        region: config.getAWSRegion(),
    },
    logLevel: config.getProperty('LOG_LEVEL'),
};

// Data warehouse API stack
const apiStack = new APIStack(app, `dw-api`, {
    stackName: `${config.getAudience()}-dw-api`,
    ...props,
});
config.addTags(apiStack);

// SQS stack
const dataEventProcessorStack = new DataEventProcessorStack(app, 'data-event-processor-stack', {
    noProxy: config.getNoProxy(),
    stackName: `${config.getAudience()}-data-event-processor-stack`,
    lambdaSGId: securityStack.getLambdaSG().securityGroupId,
    apiGateway: apiStack.getApiGateway(),
    ...props,
});

Error Log

Error: 'data-event-processor-stack' depends on 'dw-api' (data-event-processor-stack/queue-api/Resource -> dw-api/DWAPIGateway/Resource.RootResourceId). Adding this dependency (dependency added using stack.addDependency()) would create a cyclic reference.
    at APIStack.addDependency (/home/INTL/dw/modules/common-infra/api-stack/node_modules/@aws-cdk/core/lib/stack.ts:283:15)
    at APIStack.prepare (/home/INTL/dw/modules/common-infra/api-stack/node_modules/@aws-cdk/core/lib/stack.ts:509:14)
    at Function.prepare (/home/INTL/dw/infrastructure/node_modules/@aws-cdk/core/lib/construct.ts:84:28)
    at Function.synth (/home/INTL/dw/infrastructure/node_modules/@aws-cdk/core/lib/construct.ts:41:10)
    at App.synth (/home/INTL/dw/infrastructure/node_modules/@aws-cdk/core/lib/app.ts:140:36)
    at process.App.process.once (/home/INTL/dw/infrastructure/node_modules/@aws-cdk/core/lib/app.ts:119:45)
    at Object.onceWrapper (events.js:277:13)
    at process.emit (events.js:189:13)
    at process.EventEmitter.emit (domain.js:441:20)
    at process.emit (/home/INTL/dw/node_modules/source-map-support/source-map-support.js:465:21)

Environment

• CLI Version : 1.10.1
• Framework Version: [email protected]
• OS : Linux cent OS
• Language : Typescript

Other

---

This is 🐛 Bug Report

[m-mayank]

+1, I am also facing the same issue.

[ashwgupt]

I am also facing the similar issue.

[ashwgupt]

@nija-at any possibility to look into this issue on priority? The more important things is to know, if it's really a bug or a wrong usage from our side.

[nija-at]

#4010 only fixed cyclic reference errors when APIGateway and Lambda are defined across stacks. Other cyclic reference errors can very well exist and are bugs.

I gave this a shot and the issue does arise in the way you've modeled your stacks. (For my own reference in the future - gist)

However, @praman28 - this should go away if you move the definition of the APIGateway.Resource from the 'SQS queue stack' to the 'APIGateway stack'. Can you give this a shot?

The code that works for me is below.
@ashwgupt, does yours allow for modifying something similar?

#!/usr/bin/env node
import cdk = require('@aws-cdk/core');
import apigw = require('@aws-cdk/aws-apigateway');
import sqs = require('@aws-cdk/aws-sqs');

class APIStack extends cdk.Stack {

  public resource: apigw.Resource;

  constructor(scope: cdk.Construct, id: string) {
    super(scope, id);

    const apiGateway = new apigw.RestApi(this, 'myrestapi', {
      restApiName: 'myrestapi',
    });

    apiGateway.root.addMethod('ANY');

    this.resource = new apigw.Resource(this, 'queue-api', {
      parent: apiGateway.root,
      pathPart: 'queue',
    });
  }
}

class SQSStack extends cdk.Stack {

  constructor(scope: cdk.Construct, id: string, private readonly resource: apigw.Resource) {
    super(scope, id);
    const queue = new sqs.Queue(this, 'myqueue', {
      queueName : 'myqueue'
    });

    const queueIntegration = new apigw.AwsIntegration({
      service: 'SQS',
      integrationHttpMethod: 'POST',
      path: queue.queueName,
    });

    this.resource.addMethod('POST', queueIntegration);
  }
}

const app = new cdk.App();
const apiStack = new APIStack(app, 'myapistack');
new SQSStack(app, 'mysqsstack', apiStack.resource);
app.synth();

[praman28]

@nija-at Thanks for looking into this. The approach you are suggesting of keeping the APIGateway and its resource in a single stack is something i have already tried and it worked for me . What i was interested in was keeping my resources as modular as possible , in this case i always want that if i destroy my SQS then i also delete my associated resources without making any changes to APIGateway. Basically i am looking for something that works for APIGateway and Lambda and was fixed in #4010 . I'll wait for the resolution to come and meanwhile use workarounds for my code.

[ashwgupt]

@nija-at yes, this works as expected but we do want to maintain the modularity of our stacks in the bounded context of our microservice and hence would need to do it in a way that, API resources live with the API backend (microservice).

[nija-at]

@praman28 & @ashwgupt - I wanted to clarify a couple of things.

1. The fix here might very well turn out that under the hood CDK would do the same thing as suggested, i.e., move the apigateway.Resource construct into the other stack to avoid cyclic dependency.
   What this means is that while your CDK app code might look as though Queue is sitting in one stack while RestApi and Resource are sitting in another stack, the generated CDK generated CloudFormation templates will probably some of the APIGateway constructs moving to the other stack.
   In fact, this is exactly what fixed the APIGateway Lambda cyclic dependency bug you've both referenced (fix(apigateway): cross-stack lambda integration causes a cyclic reference #4010). Take a look at the test and the generated CF template to get an idea.
   Just to clarify, I'm not suggesting that this is how this issue will get fixed but that it is a likely outcome. There may be other ways to model these two stacks without it being circular.

2. I'm not entirely clear what modularity you lose with this setup. Why is it more modular to expose an instance of the RestApi rather than expose a specific Resource? Could you expand on this?
   It could be argued that exposing the RestApi is less desirable and less safe - it could be mutated in ways that you don't expect it to be in other stacks. OTOH, exposing a specific Resource means that the level of control you're exposing outside of that one stack is limited.

[papertradr]

#4010 only fixed cyclic reference errors when APIGateway and Lambda are defined across stacks. Other cyclic reference errors can very well exist and are bugs.

I gave this a shot and the issue does arise in the way you've modeled your stacks. (For my own reference in the future - gist)

However, @praman28 - this should go away if you move the definition of the APIGateway.Resource from the 'SQS queue stack' to the 'APIGateway stack'. Can you give this a shot?

The code that works for me is below.
@ashwgupt, does yours allow for modifying something similar?

#!/usr/bin/env node
import cdk = require('@aws-cdk/core');
import apigw = require('@aws-cdk/aws-apigateway');
import sqs = require('@aws-cdk/aws-sqs');

class APIStack extends cdk.Stack {

  public resource: apigw.Resource;

  constructor(scope: cdk.Construct, id: string) {
    super(scope, id);

    const apiGateway = new apigw.RestApi(this, 'myrestapi', {
      restApiName: 'myrestapi',
    });

    apiGateway.root.addMethod('ANY');

    this.resource = new apigw.Resource(this, 'queue-api', {
      parent: apiGateway.root,
      pathPart: 'queue',
    });
  }
}

class SQSStack extends cdk.Stack {

  constructor(scope: cdk.Construct, id: string, private readonly resource: apigw.Resource) {
    super(scope, id);
    const queue = new sqs.Queue(this, 'myqueue', {
      queueName : 'myqueue'
    });

    const queueIntegration = new apigw.AwsIntegration({
      service: 'SQS',
      integrationHttpMethod: 'POST',
      path: queue.queueName,
    });

    this.resource.addMethod('POST', queueIntegration);
  }
}

const app = new cdk.App();
const apiStack = new APIStack(app, 'myapistack');
new SQSStack(app, 'mysqsstack', apiStack.resource);
app.synth();

This unfortunately did not work on python. I still get the cyclic dependencies despite creating all resources in the api gateway stack.

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.