Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Layer code update not reflected on Lambda Function #3540

Closed
Halembek opened this issue Oct 30, 2021 · 15 comments
Closed

Layer code update not reflected on Lambda Function #3540

Halembek opened this issue Oct 30, 2021 · 15 comments
Labels
maintainer/need-followup stage/bug-repro The issue/bug needs to be reproduced

Comments

@Halembek
Copy link

Halembek commented Oct 30, 2021

Description:

Changes made to layer code are not reflected as a result of deploying a new version of the layer using sam deploy

Steps to reproduce:

  • Create a layer
  • Create a group of lambda functions that depends on the created layer
  • After deploying for the first time, update just the code of the layer, and make no changes to the Lambdas
  • Deploy using sam deploy, since no changes were made to the lambdas just a new version of the layer is published and deployed

Observed result:

  • As a result the new version of the layer is assigned to existing lambdas but the updated code will not be reflected even when the new version of the layer is deployed and assigned to the lambdas

Expected result:

Changes of the code in the new layer version should be reflected in the lambdas even if no new version of the lambdas was deployed

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Windows 10
  2. sam --version: 1.28.0

Workaround

  • Manually deploy a new versión of the lambda function using Aws Console and the changes to the layer will be reflected in the lambda functionality
  • Another option would be to "force" the lambdas update/deploy by adding a comment or something that indicates sam that the function should be redeployed, nevertheless it is not clear in the documentation how sam decides that a resource should be updated in the stack
@ssenchenko ssenchenko added stage/needs-investigation Requires a deeper investigation and removed maintainer/need-response labels Dec 9, 2021
@ssenchenko ssenchenko added stage/bug-repro The issue/bug needs to be reproduced and removed stage/needs-investigation Requires a deeper investigation maintainer/need-response labels Dec 13, 2021
@ssenchenko
Copy link
Contributor

So sorry you encountered a bug! Thanks for reporting the issue, we will be investigating it further.

@erashdan
Copy link

erashdan commented May 11, 2022

The issue is only if the lambda is provisioned with alias

AutoPublishAlias: live
ProvisionedConcurrencyConfig:
        ProvisionedConcurrentExecutions: 1
DeploymentPreference:
        Type: AllAtOnce
        Enabled: true

@timhaines
Copy link

@ssenchenko It's been 7 months. Any update?

@joseph-prezent
Copy link

Facing same issue for a while now, any updates or workaround which can be used?

@atlesp
Copy link

atlesp commented Sep 30, 2022

We are facing the same issue with layers and we had a long call with AWS support yesterday to get our production environment to work again due to this bug. We ended up bumping the Memory setting on all the Lambda to be sure they used the latest version.

We also use AutoPublishAlias as mentioned earlier in this thread.

What we saw happening was as follows:

  1. The Lambda config gets updated with the new layer version
  2. but a new version of the Lambda is not created
  3. Hence the alias is this pointing to the "old" version

Then all our API requests coming in from API Gateway of course still are pointing to the live alias that points to the "old" version.

We tried to add the layer version as an environment variable in all the lambdas to force an update, but then we hit the issue described here. It could also look like it is the same issue itself since we are sending the layer version ARN as a param to the template

@saidaisri
Copy link

As per erashdan comments, the issue is only with lambda provisioned capacity used along with alias.
New lambda version is not published when dependent lambda layer is updated.

AutoPublishAlias: live
  ProvisionedConcurrencyConfig:
	ProvisionedConcurrentExecutions: 5

Workaround to force publish new version of lambda from atlesp comment fixed the issue.
Added new dummy environment variable with value as logical ID of Layer, for lambdas using respective layers.

Environment:
	Variables:
	  LAYER_SHARED_VERSION: !Ref shared

!Ref shared is the reference for AWS::Serverless::LayerVersion module.

@thomasjetzinger
Copy link

@saidaisri

This doesn't work for me? I always have to create new version manually and update the alias with the latest version

@PritamDutt
Copy link

PritamDutt commented Aug 21, 2023

I am also struggling with same issue... 😞

This only appears to work, when you introduce the new variable into the configuration, but would not create any more versions after that

As per erashdan comments, the issue is only with lambda provisioned capacity used along with alias. New lambda version is not published when dependent lambda layer is updated.

AutoPublishAlias: live
  ProvisionedConcurrencyConfig:
	ProvisionedConcurrentExecutions: 5

Workaround to force publish new version of lambda from atlesp comment fixed the issue. Added new dummy environment variable with value as logical ID of Layer, for lambdas using respective layers.

Environment:
	Variables:
	  LAYER_SHARED_VERSION: !Ref shared

!Ref shared is the reference for AWS::Serverless::LayerVersion module.

Workaround to force publish new version of lambda from atlesp comment fixed the issue. Added new dummy environment variable with value as logical ID of Layer, for lambdas using respective layers.

Environment:
	Variables:
	  LAYER_SHARED_VERSION: !Ref shared

!Ref shared is the reference for AWS::Serverless::LayerVersion module.

@mpoplavkov
Copy link

I encounter the same issue. I have some lambdas with provisioned concurrency and some without. I tried to update the layer both types depend on. After publishing (using sam deploy) only the functions without provisioned concurrency reflect the layer updates. The ones with provisioned concurrency I have to deploy manually via AWS console

@dannosaur
Copy link

dannosaur commented Jan 17, 2024

Adding my $0.02 here. I've just stumbled across this issue as well, exactly as described. My AWS folks suggested to write my own Alias and Version resources to override the ones SAM generates to effectively force it, but that appears to require also writing a custom Permission resource, and then overriding the RestApi resource by hand to invoke the Alias instead of the function.

Still an issue in SAM CLI 1.107.0.

@hawflau
Copy link
Contributor

hawflau commented Feb 9, 2024

For those who are still having the issue, does it only happen when AutoPublishAlias is used in your AWS::Serverless::Function resources?

@hawflau
Copy link
Contributor

hawflau commented Feb 9, 2024

Hey all, I tried to reproduce the issue and it seems it's due to AutoPublishAlias. Then I also set AutoPublishAliasAllProperties: true in the Lambda Function, which forces it to create a new Lambda Version and update the Alias.

It is not a SAM CLI issue. Transferring to SAM-T repo

@hawflau hawflau transferred this issue from aws/aws-sam-cli Feb 9, 2024
@paulhcsun
Copy link
Contributor

As mentioned by @hawflau, the solution is to set AutoPublishAliasAllProperties: true in the Lambda Function. Closing this issue.

Copy link
Contributor

github-actions bot commented Feb 9, 2024

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.

@niklas-palm
Copy link

This issue is still prevalent if you're not using AutoPublishAlias

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintainer/need-followup stage/bug-repro The issue/bug needs to be reproduced
Projects
None yet
Development

No branches or pull requests