Cannot pull or switch between environments containing notifications #6475

jcbdev · 2021-01-25T09:37:29Z

Describe the bug
I opened an issue yesterday on this but I was looking in the wrong place #6469 and I also have experienced this before in #5056. Both times I thought it was old issue or a config issue but there is definitely a recurring problem that breaks my CI pipeline on a regular basis.

I think the reason this is occurring due to one particular value in amplify-meta.json not being updated correctly. I think it pulls from the first environment in team-provider-info.json instead of the environment you are trying to initialise. This means if you destroy the other environment the new environment is broken because it cannot update the APNS/FCM resources for wrong and old environment and thus prevents you switching or pulling environment.

This is the amplify-meta.json of a newly created environment from my machine. I had one environment called "core" before running init again. After creating the environment and then downloading the amplify-meta.json from the deploy bucket you can see the value below still points the pinpoint appid of the "core" environment and not the new appid.

If I now teardown "core" then this new environment is now completely broken (cannot pull or checkout). To correct I have to edit the amplify-meta.json in the #current-cloud-backend.zip and reupload it just to fix it

"notifications": {
    "myNotifications": {
      "service": "Pinpoint",
      "output": {
        "Arn": "arn:aws:mobiletargeting:eu-west-1:<account>:apps/acdb82ba838543e6b4ccf86ccef3a4e2",
        "Id": "acdb82ba838543e6b4ccf86ccef3a4e2",
        "Name": "myNotifications-newenv",
        "tags": {},
        "Region": "eu-west-1",
        "APNS": {
          "ApplicationId": "acdb82ba838543e6b4ccf86ccef3a4e2",
          "CreationDate": "2021-01-24T20:38:49.442Z",
          "DefaultAuthenticationMethod": "Certificate",
          "Enabled": true,
          "HasCredential": true,
          "HasTokenKey": false,
          "Id": "apns",
          "IsArchived": false,
          "LastModifiedDate": "2021-01-24T20:38:49.442Z",
          "Platform": "APNS",
          "Version": 1
        },
        "FCM": {
          "ApplicationId": "acdb82ba838543e6b4ccf86ccef3a4e2",
          "CreationDate": "2021-01-24T20:38:50.749Z",
          "Enabled": true,
          "HasCredential": true,
          "Id": "gcm",
          "IsArchived": false,
          "LastModifiedDate": "2021-01-24T20:38:50.749Z",
          "Platform": "GCM",
          "Version": 1
        },
        "Email": {
          "ApplicationId": "acdb82ba838543e6b4ccf86ccef3a4e2",
          "CreationDate": "2021-01-24T20:38:51.162Z",
          "Enabled": true,
          "FromAddress": "[email protected]",
          "Id": "email",
          "Identity": "arn:aws:ses:eu-west-1:<account>:identity/email.co.uk",
          "IsArchived": false,
          "LastModifiedDate": "2021-01-24T20:38:51.162Z",
          "MessagesPerSecond": 1,
          "Platform": "EMAIL",
          "RoleArn": "arn:aws:iam::<account>:role/service-role/pinpoint-events",
          "Version": 1
        },
        "SMS": {
          "ApplicationId": "acdb82ba838543e6b4ccf86ccef3a4e2",
          "CreationDate": "2021-01-24T20:38:51.795Z",
          "Enabled": true,
          "Id": "sms",
          "IsArchived": false,
          "LastModifiedDate": "2021-01-24T20:38:51.795Z",
          "Platform": "SMS",
          "PromotionalMessagesPerSecond": 20,
          "TransactionalMessagesPerSecond": 20,
          "Version": 1
        }
      },
      "lastPushTimeStamp": "2021-01-24T20:38:47.919Z"
    }
  },
  "analytics": {
    "myNotifications": {
      "service": "Pinpoint",
      "providerPlugin": "awscloudformation",
      "dependsOn": [
        {
          "category": "auth",
          "resourceName": "myvipapp",
          "attributes": [
            "UserPoolId"
          ]
        },
        {
          "category": "auth",
          "resourceName": "userPoolGroups",
          "attributes": [
            "adminGroupRole"
          ]
        },
        {
          "category": "auth",
          "resourceName": "userPoolGroups",
          "attributes": [
            "publicGroupRole"
          ]
        }
      ],
      "providerMetadata": {
        "s3TemplateURL": "https://s3.amazonaws.com/amplify-myapp-newenv-134601-deployment/amplify-cfn-templates/analytics/pinpoint-cloudformation-template.json",
        "logicalId": "analyticsmyNotifications"
      },
      "lastPushTimeStamp": "2021-01-24T21:04:04.080Z",
      "output": {
        "appName": "myNotifications", <----- NOT CHANGED FROM CORE SETTINGS
        "Region": "eu-west-1",
        "Id": "c40b2b7d9baa475b811fd07d74240084" <--- NOT CHANGED FROM CORE SETTINGS
      },
      "lastPushDirHash": "19mkE0kFkhhdMb2VC5pxjiHSuKA="
    }
  },

Amplify CLI Version
4.41.2

To Reproduce
See above

Expected behavior
This value should be updated to the current environment settings from team-provider-info.json

Desktop (please complete the following information):

OS: Mac or Linux
Node Version. v14.4.0

Log output
None really

2021-01-25T09:00:15.257Z|info : amplify env core checkout eventgroup 
2021-01-25T09:03:51.334Z|info : amplify pull core  [***]{"amplify":"{\"projectName\":\"[***]App\",\"defaultEditor\":\"code\",\"envName\":\"[***]roup\",\"appId\":\"[***]tln025\"}","frontend":"{\"frontend\":\"javascript\",\"framework\":\"react\",\"config\":{\"SourceDir\":\"packages/core\",\"DistributionDir\":\"packages/core/lib\",\"BuildCommand\":\"yarn build\",\"StartCommand\":\"yarn start\"}}","providers":"{\"awscloudformation\":{\"configLevel\":\"project\",\"useProfile\":true,\"profileName\":\"[***]ar-[***]ify\"\n}}","categories":"{\"notifications\":{\"Pinpoint\":{\n\"SMS\":{\n\"Enabled\":true},\"Email\":{\n\"Enabled\":true,\"FromAddress\":\"[***]elopment.co.uk\",\"Identity\":\"[***]ses:eu-[***]st-[***]5:identity/[***]nt.co.uk\",\"RoleArn\":\"[***]6357915:role/[***]ice-[***]le/[***]oint-[***]nts\"},\"APNS\":{\n\"Enabled\":true,\"DefaultAuthenticationMethod\":\"Certificate\",\"P12FilePath\":\"/Users/jimbo/Projects/roar/certs/push-notifications.p12\",\"P12FilePassword\":\"\"},\"FCM\":{\n\"Enabled\":true,\"ApiKey\":\"[***]Mh3xZwmxZVKZFWyBh2OJaZY0cYe-[***]9Pw4FSEn9vneq-[***]BJ90wA4RHBPWWoN7YlCiW\"}}},\"auth\":{\"facebookAppId\":\"[***]130007\",\"googleClientId\":\"[***]55857-[***]ps.googleusercontent.com\",\"facebookAppIdUserPool\":\"[***]130007\",\"facebookAppSecretUserPool\":\"[***]99fde78abbfbb\",\"googleAppIdUserPool\":\"[***]55857-[***]ps.googleusercontent.com\",\"googleAppSecretUserPool\":\"[***]QgWP2XF8IH\"}}","yes":true,"restore":true}
2021-01-25T09:03:57.567Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:03:57.567Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:03:57.569Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:03:57.569Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:03:57.570Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"eu-west-1"}])
2021-01-25T09:03:57.570Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["roar-amplify"])
2021-01-25T09:03:57.570Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/credentials"])
2021-01-25T09:03:57.576Z|info : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
2021-01-25T09:03:58.051Z|info : amplify-provider-awscloudformation.attach-backend.getAmplifyApp.amplifyClient.getApp([{"appId":"[***]tln025"}])
2021-01-25T09:03:58.452Z|info : amplify-provider-awscloudformation.attach-backend.getBackendEnv.amplifyClient.getBackendEnvironment([{"appId":"[***]tln025","environmentName":"[***]roup"}])
2021-01-25T09:03:58.654Z|info : amplify-provider-awscloudformation.attach-backend.downloadBackend.s3.getObject([{"Key":"[***]rent-[***]ud-[***]d.zip","Bucket":"[***]ify-[***]app-[***]roup-[***]601-[***]ment"}])
2021-01-25T09:04:02.855Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:04:02.855Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:04:02.856Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"eu-west-1"}])
2021-01-25T09:04:02.856Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["roar-amplify"])
2021-01-25T09:04:02.856Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/credentials"])
2021-01-25T09:04:02.856Z|info : amplify-provider-awscloudformation.zip-util.downloadZip.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip"},null])
2021-01-25T09:04:02.858Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip","Bucket":"[***]ify-[***]app-[***]roup-[***]601-[***]ment"}])
2021-01-25T09:04:03.876Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:04:03.876Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:04:03.876Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"eu-west-1"}])
2021-01-25T09:04:03.876Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["roar-amplify"])
2021-01-25T09:04:03.877Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/credentials"])
2021-01-25T09:04:03.881Z|info : amplify-provider-awscloudformation.initialize-env.run.cfn.updateamplifyMetaFileWithStackOutputs([{"StackName":"[***]ify-[***]app-[***]roup-[***]601"}])
2021-01-25T09:04:04.686Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]ify-[***]json","Bucket":"[***]ify-[***]app-[***]roup-[***]601-[***]ment"}])
2021-01-25T09:04:04.854Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:04:04.854Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:04:04.854Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"eu-west-1"}])
2021-01-25T09:04:04.854Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["roar-amplify"])
2021-01-25T09:04:04.855Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/credentials"])
2021-01-25T09:04:06.393Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:04:06.393Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/config"])
2021-01-25T09:04:06.394Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"eu-west-1"}])
2021-01-25T09:04:06.394Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["roar-amplify"])
2021-01-25T09:04:06.394Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/Users/jimbo/.aws/credentials"])
2021-01-25T09:04:06.534Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["roar-amplify"])
2021-01-25T09:20:48.071Z|info : amplify version core  {"version":true,"yes":false}

The text was updated successfully, but these errors were encountered:

jcbdev · 2021-01-25T10:30:11Z

It might be that this value never gets updated (instead of getting pulled from team-provider-info.json). So if you create an environment from another environment then the value propagates to each environment without being updated.

jcbdev · 2021-01-25T17:55:37Z

I have tracked down the "source" of the old app id on new environments. It is pulled from the appId in the parameters.json in the folder alongside the cloudformation file

for example in file amplify/backend/analytics/myNotifications/parameters.json

This appId at the bottom never gets updated and thus the value in the amplify-meta.json is always pointing to the wrong pinpoint.

{
  "appName": "myNotifications",
  "roleName": "pinpointLambdaRole5b86667f",
  "cloudformationPolicyName": "cloudformationPolicy5b86667f",
  "cloudWatchPolicyName": "cloudWatchPolicy5b86667f",
  "pinpointPolicyName": "pinpointPolicy5b86667f",
  "authPolicyName": "pinpoint_amplify_5b86667f",
  "unauthPolicyName": "pinpoint_amplify_5b86667f",
  "authRoleName": {
    "Ref": "AuthRoleName"
  },
  "unauthRoleName": {
    "Ref": "UnauthRoleName"
  },
  "authRoleArn": {
    "Fn::GetAtt": ["AuthRole", "Arn"]
  },
  "appId": "c40b2b7d9baa475b811fd07d74240084"
}

Not only does the appId not change but none of these resources exist anymore as the environment is long torn down.

jcbdev · 2021-02-10T03:54:21Z

@attilah @renebrandel

I am looking at the PR for this change and could you explain something to me?

Does this change mean you can only have one pinpoint app? Because multiple pinpoint apps was working before if you manually updated those two values in my first message. Is there a reason you have only forced the one environment?

Also, I have sandboxes for feature branches and I think looking at it that if I teardown one environment it will still remove the pinpoint application and thus destroy all other environments.

This actually makes the situation worse over the original?

jcbdev · 2021-02-10T10:29:38Z

This still isnt working correctly on the latest version

When I created a new environment it created a new pinpoint app but left the amplify-meta.json pointing to the original environment. So I was in environment "master" and run a headless script to create environment "test". There was now two apps called "myNotifications-master" and "myNotification-test" but the meta still points to "myNotifications-master". When I then called amplify env remove test it tore down the test environment except it left "myNotifications-test" and deleted "myNotifications-master" app. Thus breaking the master environment!

jcbdev · 2021-02-25T07:41:13Z

@attilah @renebrandel @kaustavghosh06 This is NOT fixed and is much worse!

To workaround this I have had to put this script in for post init of a new environment (and then re-push). Otherwise the values are still wrong and the original (master env for me) pinpoint gets torn down when the new environment is torn down and the new pinpoint gets left instead

// fix for pinpoint multienv bug - https://github.com/aws-amplify/amplify-cli/issues/6475
const notifcationsParams = JSON.parse(
  fs.readFileSync(path.resolve('../amplify/backend/analytics/myNotifications/parameters.json')).toString(),
);
const meta = JSON.parse(fs.readFileSync(path.resolve('../amplify/backend/amplify-meta.json')).toString());

const pinpoint = new AWS.Pinpoint({
  region: global.env.ANALYTICS_myNOTIFICATIONS_REGION,
});

const appIds: PromiseResult<AWS.Pinpoint.GetAppsResponse, AWS.AWSError> = await pinpoint.getApps({}).promise();

const pinpointAppName = `myNotifications-master-${global.env.ENV}`;
const pinpointAppId = (appIds.$response.data as AWS.Pinpoint.GetAppsResponse).ApplicationsResponse.Item.filter(
  a => a.Name == pinpointAppName,
);
notifcationsParams.appName = pinpointAppName;
notifcationsParams.appId = pinpointAppId[0].Id;
console.log('Updating parameter.json for analytics/myNotifications');
fs.writeFileSync(
  path.resolve('../amplify/backend/analytics/myNotifications/parameters.json'),
  JSON.stringify(notifcationsParams, null, 2),
);

console.log('Updating team-provider-info.json for missing pinpoint info');
let teamProvider = JSON.parse(fs.readFileSync(path.resolve('../amplify/team-provider-info.json')).toString());
teamProvider[global.env.ENV].categories.notifications.Pinpoint.Name = pinpointAppName;
teamProvider[global.env.ENV].categories.notifications.Pinpoint.Id = pinpointAppId[0].Id;
fs.writeFileSync(path.resolve('../amplify/team-provider-info.json'), JSON.stringify(teamProvider, null, 2));

console.log('Patching amplify meta data');
meta.notifications['myNotifications-master'].output.Name = pinpointAppName;
meta.notifications['myNotifications-master'].output.Id = pinpointAppId[0].Id;
meta.notifications['myNotifications-master'].output.APNS.Id = pinpointAppId[0].Id;
meta.notifications['myNotifications-master'].output.FCM.Id = pinpointAppId[0].Id;
meta.notifications['myNotifications-master'].output.Email.Id = pinpointAppId[0].Id;
meta.analytics.myNotifications.output.Name = pinpointAppName;
meta.analytics.myNotifications.output.Id = pinpointAppId[0].Id;
fs.writeFileSync(path.resolve('../amplify/backend/analytics/myNotifications/parameters.json'), JSON.stringify(teamProvider, null, 2));

* fix: pull issue with multi-env notifications #6475 * fix notification category multi-env handling * fix to ask for `API key` for FCM on `remove notification` command * add E2E for notification testing

github-actions · 2021-05-25T01:28:29Z

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels for those types of questions.

attilah self-assigned this Jan 25, 2021

attilah added bug Something isn't working notifications Issues tied to the notifications category labels Jan 25, 2021

attilah mentioned this issue Feb 1, 2021

fix: pull issue with multi-env notifications #6475 #6525

Merged

attilah linked a pull request Feb 1, 2021 that will close this issue

fix: pull issue with multi-env notifications #6475 #6525

Merged

attilah closed this as completed in #6525 Mar 2, 2021

github-actions bot locked as resolved and limited conversation to collaborators May 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot pull or switch between environments containing notifications #6475

Cannot pull or switch between environments containing notifications #6475

jcbdev commented Jan 25, 2021

jcbdev commented Jan 25, 2021 •

edited

Loading

jcbdev commented Jan 25, 2021

jcbdev commented Feb 10, 2021 •

edited

Loading

jcbdev commented Feb 10, 2021

jcbdev commented Feb 25, 2021

github-actions bot commented May 25, 2021

Cannot pull or switch between environments containing notifications #6475

Cannot pull or switch between environments containing notifications #6475

Comments

jcbdev commented Jan 25, 2021

jcbdev commented Jan 25, 2021 • edited Loading

jcbdev commented Jan 25, 2021

jcbdev commented Feb 10, 2021 • edited Loading

jcbdev commented Feb 10, 2021

jcbdev commented Feb 25, 2021

github-actions bot commented May 25, 2021

jcbdev commented Jan 25, 2021 •

edited

Loading

jcbdev commented Feb 10, 2021 •

edited

Loading