Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add TargetTrackingScaling as a scaling policy option #771

Conversation

falcon78921
Copy link
Contributor

SUMMARY

Add TargetTrackingScaling as an EC2 scaling option.
Fixes: #544

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

ec2_scaling_policy.py

@ansibullbot
Copy link

@ansibullbot ansibullbot added WIP Work in progress feature This issue/PR relates to a feature request module module needs_triage new_contributor Help guide this first time contributor plugins plugin (any type) labels Oct 19, 2021
@@ -272,6 +273,13 @@ def create_scaling_policy(connection, module):
if module.params['cooldown']:
params['Cooldown'] = module.params['cooldown']

if policy_type == 'TargetTrackingScaling':
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@falcon78921
Copy link
Contributor Author

falcon78921 commented Nov 16, 2021

As it is now, predefined_metric_spec seems to work:

  tasks:
   - name: create policy
     ec2_scaling_policy:
      name: "test123"
      policy_type: TargetTrackingScaling
      target_tracking_config:
        predefined_metric_spec:
          predefined_metric_type: ASGAverageCPUUtilization
        target_value: 125.5
      asg_name: "asg-test-1"
changed: [127.0.0.1] => {
    "alarms": [
        {
...
...
...
    "as_name": "asg-test-1",
    "auto_scaling_group_name": "asg-test-1",
    "changed": true,
    "enabled": true,
    "invocation": {
        "module_args": {
            "adjustment_type": null,
            "asg_name": "asg-test-1",
            "aws_access_key": null,
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": null,
            "cooldown": null,
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "estimated_instance_warmup": null,
            "metric_aggregation": "Average",
            "min_adjustment_step": null,
            "name": "test123",
            "policy_type": "TargetTrackingScaling",
            "profile": null,
            "region": null,
            "scaling_adjustment": null,
            "security_token": null,
            "state": "present",
            "step_adjustments": null,
            "target_tracking_config": {
                "customized_metric_spec": null,
                "predefined_metric_spec": {
                    "predefined_metric_type": "ASGAverageCPUUtilization"
                },
                "target_value": 125.5
            },
            "validate_certs": true
        }
    },
    "name": "test123",
...
...
...
    "policy_name": "test123",
    "policy_type": "TargetTrackingScaling",
    "step_adjustments": [],
    "target_tracking_configuration": {
        "disable_scale_in": false,
        "predefined_metric_specification": {
            "predefined_metric_type": "ASGAverageCPUUtilization"
        },
        "target_value": 125.5
    }
}

Copy link
Collaborator

@jillr jillr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good progress on this, thanks @falcon78921!

plugins/modules/ec2_scaling_policy.py Outdated Show resolved Hide resolved
plugins/modules/ec2_scaling_policy.py Outdated Show resolved Hide resolved
plugins/modules/ec2_scaling_policy.py Outdated Show resolved Hide resolved
plugins/modules/ec2_scaling_policy.py Outdated Show resolved Hide resolved
module.fail_json(msg='step_adjustments is required when policy_type is StepScaling '
'and state is present')
module.fail_json(msg='step_adjustments is required when policy_type is StepScaling'
'and state is present')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like your editor has "format on save" enabled, this creates a lot of unnecessary diff noise which can be difficult to review (now, and if we need to go through git history later for bugfixes). would you be able to toggle this setting off for this repo and back out the formatting changes in this file please?

plugins/modules/ec2_scaling_policy.py Outdated Show resolved Hide resolved
@ansibullbot ansibullbot added integration tests/integration tests tests and removed new_contributor Help guide this first time contributor labels Feb 1, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build failed (third-party-check pipeline) integration testing with
OpenStack. For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

@softwarefactory-project-zuul
Copy link
Contributor

Build failed (third-party-check pipeline) integration testing with
OpenStack. For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

@falcon78921
Copy link
Contributor Author

ansible-test-units-community-aws-python38 fails here:

2022-02-01 14:55:14.762412 | controller | [gw0] [ 97%] FAILED tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_check_for_update_tags
2022-02-01 14:55:15.142097 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_check_for_update_nonmodifiable_attr
2022-02-01 14:55:15.143054 | controller | [gw0] [ 98%] FAILED tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_check_for_update_nonmodifiable_attr
2022-02-01 14:55:15.529054 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_add_tags
2022-02-01 14:55:15.530078 | controller | [gw0] [ 99%] FAILED tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_add_tags
2022-02-01 14:55:16.143226 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_remove_tags
2022-02-01 14:55:16.224015 | controller | [gw0] [100%] FAILED tests/unit/plugins/modules/test_ec2_vpc_vpn.py::test_remove_tags
2022-02-01 14:55:16.224050 | controller |
2022-02-01 14:55:16.224054 | controller | =================================== FAILURES ===================================
2022-02-01 14:55:16.224058 | controller | _______________________________ test_upload_api ________________________________
2022-02-01 14:55:16.224061 | controller | [gw0] linux -- Python 3.8.12 /home/zuul/venv/bin/python
2022-02-01 14:55:16.224064 | controller | monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff8e611bf70>
2022-02-01 14:55:16.224068 | controller |
2022-02-01 14:55:16.224071 | controller |     def test_upload_api(monkeypatch):
2022-02-01 14:55:16.224074 | controller |         class FakeConnection:
2022-02-01 14:55:16.224077 | controller |
2022-02-01 14:55:16.224080 | controller |             def put_rest_api(self, *args, **kwargs):
2022-02-01 14:55:16.224083 | controller |                 assert kwargs["body"] == "the-swagger-text-is-fake"
2022-02-01 14:55:16.224086 | controller |                 return {"msg": "success!"}
2022-02-01 14:55:16.224090 | controller |
2022-02-01 14:55:16.224093 | controller |         def return_fake_connection(*args, **kwargs):
2022-02-01 14:55:16.224096 | controller |             return FakeConnection()
2022-02-01 14:55:16.224099 | controller |
2022-02-01 14:55:16.224102 | controller | >       monkeypatch.setattr(core, "boto3_conn", return_fake_connection)
2022-02-01 14:55:16.224106 | controller | E       AttributeError: <module 'ansible_collections.amazon.aws.plugins.module_utils.core' (<ansible.utils.collection_loader.AnsibleCollectionLoader object at 0x7ff8ec5fd970>)> has no attribute 'boto3_conn'
2022-02-01 14:55:16.224110 | controller |
2022-02-01 14:55:16.224113 | controller | tests/unit/plugins/modules/test_aws_api_gateway.py:44: AttributeError
2022-02-01 14:55:16.224116 | controller | _________________________ test_find_connection_filters _________________________
2022-02-01 14:55:16.224119 | controller | [gw0] linux -- Python 3.8.12 /home/zuul/venv/bin/python
2022-02-01 14:55:16.224122 | controller | placeboify = Session(region_name='us-west-2'), maybe_sleep = None
2022-02-01 14:55:16.224125 | controller |
2022-02-01 14:55:16.224133 | controller |     def test_find_connection_filters(placeboify, maybe_sleep):
2022-02-01 14:55:16.224137 | controller |         # setup dependencies for 2 vpn connections
2022-02-01 14:55:16.224140 | controller | >       dependencies = setup_req(placeboify, 2)
2022-02-01 14:55:16.224143 | controller |
2022-02-01 14:55:16.224146 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py:139:
2022-02-01 14:55:16.224149 | controller | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2022-02-01 14:55:16.224152 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py:361: in setup_req
2022-02-01 14:55:16.224155 | controller |     m, conn = setup_mod_conn(placeboify, params)
2022-02-01 14:55:16.224158 | controller | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2022-02-01 14:55:16.224161 | controller |
2022-02-01 14:55:16.224164 | controller | placeboify = Session(region_name='us-west-2')
2022-02-01 14:55:16.224167 | controller | params = {'connection_type': 'ipsec.1', 'customer_gateway_id': 'cgw-6113c87f', 'delay': 15, 'filters': {}, ...}
2022-02-01 14:55:16.224171 | controller |
2022-02-01 14:55:16.224174 | controller |     def setup_mod_conn(placeboify, params):
2022-02-01 14:55:16.224177 | controller |         conn = placeboify.client('ec2')
2022-02-01 14:55:16.224180 | controller |         retry_decorator = aws_ec2.AWSRetry.jittered_backoff()
2022-02-01 14:55:16.224183 | controller | >       wrapped_conn = aws_core._RetryingBotoClientWrapper(conn, retry_decorator)
2022-02-01 14:55:16.224186 | controller | E       AttributeError: module 'ansible_collections.amazon.aws.plugins.module_utils.core' has no attribute '_RetryingBotoClientWrapper'
2022-02-01 14:55:16.224190 | controller |
2022-02-01 14:55:16.224192 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py:86: AttributeError
2022-02-01 14:55:16.224195 | controller | __________________ test_find_connection_insufficient_filters ___________________
2022-02-01 14:55:16.224198 | controller | [gw0] linux -- Python 3.8.12 /home/zuul/venv/bin/python
2022-02-01 14:55:16.224202 | controller | placeboify = Session(region_name='us-west-2'), maybe_sleep = None
2022-02-01 14:55:16.224204 | controller |
2022-02-01 14:55:16.224207 | controller |     def test_find_connection_insufficient_filters(placeboify, maybe_sleep):
2022-02-01 14:55:16.224211 | controller |         # get list of customer gateways and virtual private gateways
2022-02-01 14:55:16.224213 | controller |         cgw, vgw = get_dependencies()
2022-02-01 14:55:16.224217 | controller |
2022-02-01 14:55:16.224220 | controller |         # create two connections with the same tags
2022-02-01 14:55:16.224223 | controller |         params = make_params(cgw[0], vgw[0], tags={'Correct': 'Tag'})
2022-02-01 14:55:16.224225 | controller |         params2 = make_params(cgw[1], vgw[1], tags={'Correct': 'Tag'})
2022-02-01 14:55:16.224228 | controller | >       m, conn = setup_mod_conn(placeboify, params)
2022-02-01 14:55:16.224231 | controller |
2022-02-01 14:55:16.224238 | controller | tests/unit/plugins/modules/test_ec2_vpc_vpn.py:171:

I don't think this is related to ec2_scaling_policy.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed (third-party-check pipeline) integration testing with
OpenStack. For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

@falcon78921 falcon78921 requested a review from jillr February 1, 2022 22:06
@falcon78921
Copy link
Contributor Author

@jillr The sanity tests finally passed! :) When running the AWS integration tests in my environment, everything checks out ok. Not sure if I'm missing something.

@ansibullbot ansibullbot added the new_plugin New plugin label Jul 4, 2022
@tremble tremble force-pushed the aws-544-scaling branch from 1895fd3 to 773e1e4 Compare July 4, 2022 17:53
@softwarefactory-project-zuul

This comment was marked as outdated.

@github-actions
Copy link

github-actions bot commented Jul 4, 2022

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

@ansibullbot ansibullbot added community_review and removed needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR new_module New module new_plugin New plugin labels Jul 4, 2022
@tremble
Copy link
Contributor

tremble commented Jul 4, 2022

recheck

@tremble tremble force-pushed the aws-544-scaling branch from 773e1e4 to d1f81ab Compare July 4, 2022 18:24
@softwarefactory-project-zuul

This comment was marked as outdated.

@softwarefactory-project-zuul

This comment was marked as resolved.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

✔️ ansible-galaxy-importer SUCCESS in 4m 05s (non-voting)
✔️ build-ansible-collection SUCCESS in 5m 04s
✔️ ansible-test-sanity-docker-devel SUCCESS in 10m 35s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 11m 57s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 10m 42s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 10m 40s
✔️ ansible-test-units-community-aws-python38 SUCCESS in 6m 28s
✔️ ansible-test-units-community-aws-python39 SUCCESS in 6m 21s
✔️ ansible-test-splitter SUCCESS in 2m 42s
✔️ integration-community.aws-1 SUCCESS in 7m 38s
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label Jul 4, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).

✔️ ansible-galaxy-importer SUCCESS in 3m 48s (non-voting)
✔️ build-ansible-collection SUCCESS in 4m 47s
✔️ ansible-test-sanity-docker-devel SUCCESS in 12m 06s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 10m 57s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 10m 43s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 9m 23s
✔️ ansible-test-units-community-aws-python38 SUCCESS in 5m 37s
✔️ ansible-test-units-community-aws-python39 SUCCESS in 6m 33s
✔️ ansible-test-splitter SUCCESS in 2m 26s
✔️ integration-community.aws-1 SUCCESS in 6m 17s
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 4be1e70 into ansible-collections:main Jul 4, 2022
@tremble tremble added the backport-4 PR should be backported to the stable-4 branch label Jul 4, 2022
@patchback
Copy link

patchback bot commented Jul 4, 2022

Backport to stable-4: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-4/4be1e701ab69fe9870aaa7304ad96823eea1ada3/pr-771

Backported as #1306

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request Jul 4, 2022
add `TargetTrackingScaling` as a scaling policy option

SUMMARY

Add TargetTrackingScaling as an EC2 scaling option.
Fixes: #544

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

ec2_scaling_policy.py

Reviewed-by: Jill R <None>
Reviewed-by: James McClune <None>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: Mark Chappell <None>
(cherry picked from commit 4be1e70)
tremble pushed a commit that referenced this pull request Jul 4, 2022
add `TargetTrackingScaling` as a scaling policy option

SUMMARY

Add TargetTrackingScaling as an EC2 scaling option.
Fixes: #544

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

ec2_scaling_policy.py

Reviewed-by: Jill R <None>
Reviewed-by: James McClune <None>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: Mark Chappell <None>
(cherry picked from commit 4be1e70)
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Jul 4, 2022
[PR #771/4be1e701 backport][stable-4] add `TargetTrackingScaling` as a scaling policy option

This is a backport of PR #771 as merged into main (4be1e70).
SUMMARY

Add TargetTrackingScaling as an EC2 scaling option.
Fixes: #544

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

ec2_scaling_policy.py

Reviewed-by: Mark Chappell <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-4 PR should be backported to the stable-4 branch community_review feature This issue/PR relates to a feature request integration tests/integration mergeit Merge the PR (SoftwareFactory) module module plugins plugin (any type) tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ec2_scaling_policy - Add support for TargetTrackingScaling policies
7 participants