Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lambda integration tests - potential test flake #1277

Closed
1 task done
tremble opened this issue Nov 18, 2022 · 3 comments · Fixed by #1279
Closed
1 task done

lambda integration tests - potential test flake #1277

tremble opened this issue Nov 18, 2022 · 3 comments · Fixed by #1279
Labels
bug This issue/PR relates to a bug has_pr module module needs_triage plugins plugin (any type)

Comments

@tremble
Copy link
Contributor

tremble commented Nov 18, 2022 •
edited
Loading

Another issue I just found: [lambda] Integration tests fail if the account is not empty. The lambda integration tests assume there are no other lambdas in the account. I just swapped my region to one I don't use.

The error was for integration testing lambda. It was an assertion error.

  - name: lambda_info | Ensure default query value is 'config' when function name
      omitted
    lambda_info:
    register: lambda_infos_query_config
    check_mode: yes
  - name: lambda_info | Assert successfull retrieval of all information
    assert:
      that:
...
      - lambda_infos_query_config.functions[0].function_name == lambda_function_name
      -

If the freshly created lambda for this test is not the first lambda in the list (because a pre-existing lambda is), then this assertion fails.

Perhaps it's just that this specific test needs to be made a bit smarter.

Originally posted by @mdavis-xyz in #924 (comment)

Issue Type

Bug Report

Component Name

lambda

Ansible Version

CI / latest

Collection Versions

CI / latest

Configuration

CI

OS / Environment

CI

Steps to Reproduce

Run lambda integration tests in an account with existing lambda

Expected Results

Run without errors

Actual Results

Assertion failure

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot
Copy link

cc @jillr @s-hertel @steynovich
click here for bot help

@ansibullbot ansibullbot added bug This issue/PR relates to a bug module module needs_triage plugins plugin (any type) labels Nov 18, 2022
@tremble tremble changed the title lambda integration tests - potetial test flake lambda integration tests - potential test flake Nov 18, 2022
@mdavis-xyz mdavis-xyz mentioned this issue Nov 18, 2022
Merged
@mdavis-xyz
Copy link
Contributor

I tried fixing this, but got stuck. My progress is in #1279.

softwarefactory-project-zuul bot pushed a commit that referenced this issue Nov 29, 2022
@mdavis-xyz
lambda - Fix flaky integration tests which assume there are no other …
737ee75 
…functions in the account (#1279)

lambda - Fix flaky integration tests which assume there are no other functions in the account

SUMMARY
The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.
Note that this fix doesn't work. One of the assertions is failing.
TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
lambda_info
ADDITIONAL INFORMATION
Fixes #1277

Reviewed-by: Mark Chappell <None>
Reviewed-by: Matthew Davis <None>
@patchback patchback bot mentioned this issue Nov 29, 2022
Merged
patchback bot pushed a commit that referenced this issue Nov 29, 2022
@mdavis-xyz @patchback
lambda - Fix flaky integration tests which assume there are no other …
c20488c 
…functions in the account (#1279)

lambda - Fix flaky integration tests which assume there are no other functions in the account

SUMMARY
The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.
Note that this fix doesn't work. One of the assertions is failing.
TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
lambda_info
ADDITIONAL INFORMATION
Fixes #1277

Reviewed-by: Mark Chappell <None>
Reviewed-by: Matthew Davis <None>
(cherry picked from commit 737ee75)
softwarefactory-project-zuul bot pushed a commit that referenced this issue Nov 29, 2022
@patchback
lambda - Fix flaky integration tests which assume there are no other …
d163b6a 
…functions in the account (#1279) (#1289)

[PR #1279/737ee750 backport][stable-5] lambda - Fix flaky integration tests which assume there are no other functions in the account

This is a backport of PR #1279 as merged into main (737ee75).
SUMMARY
The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.
Note that this fix doesn't work. One of the assertions is failing.
TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
lambda_info
ADDITIONAL INFORMATION
Fixes #1277
abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Oct 24, 2023
@tremble
Add missing changelog fragment for 1195 (ansible-collections#1277)
57b219a 
Add missing changelog fragment for 1195

SUMMARY
Forgot to add a changelog fragment for ansible-collections#1195
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
sns_topic
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug This issue/PR relates to a bug has_pr module module needs_triage plugins plugin (any type)
Projects
None yet
Milestone
No milestone
3 participants
@tremble @mdavis-xyz @ansibullbot