fix delete records without TTL #801

markuman · 2021-11-12T16:31:50Z

SUMMARY

Closes #800

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

route53

plugins/modules/route53.py

Co-authored-by: Mark Chappell <[email protected]>

kaovilai · 2021-11-12T18:38:51Z

I just tried importing locally as of https://github.com/ansible-collections/community.aws/blob/6c0e34dbe972c330f2fdea7a7bfe2ddd72f5e2da/plugins/modules/route53.py

  - name: Get openshift route53 A records to delete
    community.aws.route53:
      state: get
      aws_access_key: "{{ aws_access_key_id }}"
      aws_secret_key: "{{ aws_secret_access_key }}"
      zone: "{{ guid }}{{ subdomain_base_suffix }}"
      record: "{{ item }}"
      type: "A"
    with_items:
      - "api.cluster-{{ guid }}.{{ guid }}{{ subdomain_base_suffix }}"
      - "*.apps.cluster-{{ guid }}.{{ guid }}{{ subdomain_base_suffix }}"
    register: records
  - debug: var=records
  - name: Delete openshift route53 A records
    # community.aws.route53:
    route53:
      state: absent
      aws_access_key: "{{ aws_access_key_id }}"
      aws_secret_key: "{{ aws_secret_access_key }}"
      zone: "{{ item.set.zone }}"
      record: "{{ item.set.record }}"
      value: "{{ item.set.value }}"
      type: "{{ item.set.type }}"
      overwrite: yes
    with_items: "{{ records.results }}"

Got

The full traceback is:
Traceback (most recent call last):
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_route53_payload_xjletf5f/ansible_route53_payload.zip/ansible/modules/route53.py", line 657, in main
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_route53_payload_xjletf5f/ansible_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 288, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_route53_payload_xjletf5f/ansible_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 157, in retry_func
    raise e
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_route53_payload_xjletf5f/ansible_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 607, in _make_api_call
    request_dict = self._convert_to_request_dict(
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 655, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
  File "/usr/local/lib/python3.9/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter ChangeBatch.Changes[0].ResourceRecordSet.TTL, value: None, type: <class 'NoneType'>, valid types: <class 'int'>
Invalid type for parameter ChangeBatch.Changes[0].ResourceRecordSet.ResourceRecords, value: None, type: <class 'NoneType'>, valid types: <class 'list'>, <class 'tuple'>
failed: [localhost] (item={'changed': False, 'set': {'Name': '*.apps.cluster-<>', 'Type': 'A', 'AliasTarget': {'HostedZoneId': '<>', 'DNSName': '<>-<>.us-east-1.elb.amazonaws.com.', 'EvaluateTargetHealth': False}, 'zone': '<>.', 'hosted_zone_id': '<>', 'type': 'A', 'record': '*.apps.cluster-<>', 'ttl': None, 'identifier': None, 'weight': None, 'region': None, 'failover': None, 'health_check': None, 'alias': True, 'value': '<>-<>.us-east-1.elb.amazonaws.com.', 'values': ['<>-<>.us-east-1.elb.amazonaws.com.'], 'alias_hosted_zone_id': '<>', 'alias_evaluate_target_health': False}, 'nameservers': ['ns-143.awsdns-17.com.', 'ns-862.awsdns-43.net.', 'ns-1400.awsdns-47.org.', 'ns-1852.awsdns-39.co.uk.'], 'resource_record_sets': [{'name': '*.apps.cluster-<>', 'type': 'A', 'alias_target': {'hosted_zone_id': '<>', 'dns_name': '<>-<>.us-east-1.elb.amazonaws.com.', 'evaluate_target_health': False}}], 'invocation': {'module_args': {'state': 'get', 'aws_access_key': '<>', 'aws_secret_key': 'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER', 'zone': '<>', 'record': '*.apps.cluster-<>.<>', 'type': 'A', 'debug_botocore_endpoint_logs': False, 'validate_certs': True, 'ttl': 3600, 'alias_evaluate_target_health': False, 'retry_interval': 500, 'private_zone': False, 'wait': False, 'wait_timeout': 300, 'ec2_url': None, 'security_token': None, 'aws_ca_bundle': None, 'profile': None, 'aws_config': None, 'region': None, 'hosted_zone_id': None, 'alias': None, 'alias_hosted_zone_id': None, 'value': None, 'overwrite': None, 'identifier': None, 'weight': None, 'health_check': None, 'failover': None, 'vpc_id': None}}, 'failed': False, 'item': '*.apps.cluster-<>.<>', 'ansible_loop_var': 'item'}) => {
    "ansible_loop_var": "item",
    "boto3_version": "1.14.43",
    "botocore_version": "1.17.43",
    "changed": false,
    "invocation": {
        "module_args": {
            "alias": null,
            "alias_evaluate_target_health": false,
            "alias_hosted_zone_id": null,
            "aws_access_key": "<>",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "failover": null,
            "health_check": null,
            "hosted_zone_id": null,
            "identifier": null,
            "overwrite": true,
            "private_zone": false,
            "profile": null,
            "record": "*.apps.cluster-<>",
            "region": null,
            "retry_interval": 500,
            "security_token": null,
            "state": "absent",
            "ttl": 3600,
            "type": "A",
            "validate_certs": true,
            "value": [
                "<>-<>.us-east-1.elb.amazonaws.com."
            ],
            "vpc_id": null,
            "wait": false,
            "wait_timeout": 300,
            "weight": null,
            "zone": "<>."
        }
    },
    "item": {
        "ansible_loop_var": "item",
        "changed": false,
        "failed": false,
        "invocation": {
            "module_args": {
                "alias": null,
                "alias_evaluate_target_health": false,
                "alias_hosted_zone_id": null,
                "aws_access_key": "<>",
                "aws_ca_bundle": null,
                "aws_config": null,
                "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "debug_botocore_endpoint_logs": false,
                "ec2_url": null,
                "failover": null,
                "health_check": null,
                "hosted_zone_id": null,
                "identifier": null,
                "overwrite": null,
                "private_zone": false,
                "profile": null,
                "record": "*.apps.cluster-<>.<>",
                "region": null,
                "retry_interval": 500,
                "security_token": null,
                "state": "get",
                "ttl": 3600,
                "type": "A",
                "validate_certs": true,
                "value": null,
                "vpc_id": null,
                "wait": false,
                "wait_timeout": 300,
                "weight": null,
                "zone": "<>"
            }
        },
        "item": "*.apps.cluster-<>.<>",
        "nameservers": [
            "ns-143.awsdns-17.com.",
            "ns-862.awsdns-43.net.",
            "ns-1400.awsdns-47.org.",
            "ns-1852.awsdns-39.co.uk."
        ],
        "resource_record_sets": [
            {
                "alias_target": {
                    "dns_name": "<>-<>.us-east-1.elb.amazonaws.com.",
                    "evaluate_target_health": false,
                    "hosted_zone_id": "<>"
                },
                "name": "*.apps.cluster-<>",
                "type": "A"
            }
        ],
        "set": {
            "AliasTarget": {
                "DNSName": "<>-<>.us-east-1.elb.amazonaws.com.",
                "EvaluateTargetHealth": false,
                "HostedZoneId": "<>"
            },
            "Name": "*.apps.cluster-<>",
            "Type": "A",
            "alias": true,
            "alias_evaluate_target_health": false,
            "alias_hosted_zone_id": "<>",
            "failover": null,
            "health_check": null,
            "hosted_zone_id": "<>",
            "identifier": null,
            "record": "*.apps.cluster-<>",
            "region": null,
            "ttl": null,
            "type": "A",
            "value": "<>-<>.us-east-1.elb.amazonaws.com.",
            "values": [
                "<>-<>.us-east-1.elb.amazonaws.com."
            ],
            "weight": null,
            "zone": "<>."
        }
    },
    "msg": "Failed to update records: Parameter validation failed:\nInvalid type for parameter ChangeBatch.Changes[0].ResourceRecordSet.TTL, value: None, type: <class 'NoneType'>, valid types: <class 'int'>\nInvalid type for parameter ChangeBatch.Changes[0].ResourceRecordSet.ResourceRecords, value: None, type: <class 'NoneType'>, valid types: <class 'list'>, <class 'tuple'>"
}

markuman · 2021-11-12T18:42:39Z

@kaovilai is your branch up to date? because that should be fixed with 95560e1
and the following commits are also important, otherwise it's not immutable and may fail: daada47

kaovilai · 2021-11-12T18:45:55Z

@markuman I just replaced the entire file with https://raw.githubusercontent.com/ansible-collections/community.aws/95560e1534d13627b14a5859c69b43febeb96b1c/plugins/modules/route53.py. Same issue.

markuman · 2021-11-12T18:45:57Z

@kaovilai is your branch up to date? because that should be fixed with 95560e1 and the following commits are also important, otherwise it's not immutable and may fail: daada47

Locally your scenario passes. And it's also added to the integration test.

---
- hosts: localhost
  connection: local

  tasks:
    - name: create
      community.aws.route53:
        state: present
        zone: xn--mitlinuxwrdasnichtpassiert-ohc.de
        record: ansible.xn--mitlinuxwrdasnichtpassiert-ohc.de.
        type: A
        value: 
          - 127.0.0.1
          - 127.0.1.1
        ttl: 30
      register: ttl30
    - name: check return values
      assert:
        that:
          - ttl30.diff.resource_record_sets[0].ttl == 30
          - ttl30 is changed

    - debug:
        var: ttl30

    - name: delete without value and ttl
      community.aws.route53:
        state: absent
        zone: xn--mitlinuxwrdasnichtpassiert-ohc.de
        record: ansible.xn--mitlinuxwrdasnichtpassiert-ohc.de.
        type: A
      register: ttl30

    - debug:
        var: ttl30

kaovilai · 2021-11-12T18:46:40Z

ill try renaming files see if it's the right file.

changelogs/fragments/801-fix-delete-without-ttl.yml

kaovilai · 2021-11-12T18:49:30Z

@markuman same issue.. at 95560e1

The full traceback is:
Traceback (most recent call last):
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_mark_route53_payload_74pt5vm5/ansible_mark_route53_payload.zip/ansible/modules/mark_route53.py", line 657, in main
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_mark_route53_payload_74pt5vm5/ansible_mark_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 288, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_mark_route53_payload_74pt5vm5/ansible_mark_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 157, in retry_func
    raise e
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_mark_route53_payload_74pt5vm5/ansible_mark_route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 607, in _make_api_call
    request_dict = self._convert_to_request_dict(
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 655, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
  File "/usr/local/lib/python3.9/site-packages/botocore/validate.py", line 297, in serialize_to_request

markuman · 2021-11-12T19:02:33Z

@markuman same issue.. at 95560e1

This is not the latest. Double-check if https://github.com/ansible-collections/community.aws/pull/801/files#diff-39a0039c8f787ef918e13c9e34de717b42890a4baada9448cd80a76242bffee4R607 exists

markuman · 2021-11-12T20:14:30Z

recheck

kaovilai · 2021-11-15T18:06:55Z

So I think the record I was trying to delete was an alias type and I didn't specify it as an alias.. so. that may have caused the error.

kaovilai · 2021-11-15T18:33:55Z

Error resolved. A alias record deleted successfully using 9247faa

changelogs/fragments/801-fix-delete-without-ttl.yml

plugins/modules/route53.py

Co-authored-by: Mark Chappell <[email protected]>

tremble

LGTM

ansible-zuul

LGTM!

changed from bugfix to minor_change. Making a mandatory parameter optional doesn't feel like a 'breaking' change, and we've got 3.0.0 coming up.

fix delete records without TTL SUMMARY Closes ansible-collections#800 ISSUE TYPE Bugfix Pull Request COMPONENT NAME route53 Reviewed-by: Mark Chappell <None> Reviewed-by: Felix Fontein <None> Reviewed-by: Tiger Kaovilai <[email protected]> Reviewed-by: Markus Bergholz <[email protected]> Reviewed-by: None <None> This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections@472776e

markuman added 2 commits November 12, 2021 17:30

tias hotfix

ba29341

sanity fix

9a78545

tremble reviewed Nov 12, 2021

View reviewed changes

plugins/modules/route53.py Outdated Show resolved Hide resolved

markuman and others added 3 commits November 12, 2021 18:37

Update plugins/modules/route53.py

c8d8cb1

Co-authored-by: Mark Chappell <[email protected]>

expand integration test and value is not required when deleting records

95560e1

add changelog fragment

78bc7f7

markuman requested a review from felixfontein November 12, 2021 18:02

markuman changed the title ~~[WIP] fix delete records without TTL~~ fix delete records without TTL Nov 12, 2021

markuman added 3 commits November 12, 2021 19:10

fix diff mode when state absent and fix immutable

daada47

append changelog

5f81cb8

documentation updates

6c0e34d

markuman requested a review from tremble November 12, 2021 18:34

felixfontein previously requested changes Nov 12, 2021

View reviewed changes

changelogs/fragments/801-fix-delete-without-ttl.yml Outdated Show resolved Hide resolved

remove diff fix

29a09f4

This comment has been minimized.

Sign in to view

fix sanity

9247faa

This comment has been minimized.

Sign in to view

This was referenced Nov 15, 2021

4.x delete script fails to delete cloudformation due to route53 hostedzone migtools/mig-agnosticd#188

Closed

Ocp4 cluster destroy running ec2 instances. Code for route53 records deletion. redhat-cop/agnosticd#4236

Closed

tremble requested changes Nov 16, 2021

View reviewed changes

changelogs/fragments/801-fix-delete-without-ttl.yml Outdated Show resolved Hide resolved

plugins/modules/route53.py Outdated Show resolved Hide resolved

markuman and others added 2 commits November 22, 2021 13:41

Update changelogs/fragments/801-fix-delete-without-ttl.yml

9d34e76

Co-authored-by: Mark Chappell <[email protected]>

Update plugins/modules/route53.py

479a4fc

Co-authored-by: Mark Chappell <[email protected]>

tremble approved these changes Nov 22, 2021

View reviewed changes

tremble added the gate label Nov 22, 2021

ansible-zuul bot approved these changes Nov 22, 2021

View reviewed changes

tremble added gate and removed gate labels Nov 22, 2021

ansible-zuul bot merged commit 472776e into ansible-collections:main Nov 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix delete records without TTL #801

fix delete records without TTL #801

markuman commented Nov 12, 2021

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

kaovilai commented Nov 12, 2021

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

This comment has been minimized.

markuman commented Nov 12, 2021

This comment has been minimized.

This comment has been minimized.

kaovilai commented Nov 15, 2021

kaovilai commented Nov 15, 2021

tremble left a comment

ansible-zuul bot left a comment

fix delete records without TTL #801

fix delete records without TTL #801

Conversation

markuman commented Nov 12, 2021

SUMMARY

ISSUE TYPE

COMPONENT NAME

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

kaovilai commented Nov 12, 2021

kaovilai commented Nov 12, 2021

markuman commented Nov 12, 2021

This comment has been minimized.

markuman commented Nov 12, 2021

This comment has been minimized.

This comment has been minimized.

kaovilai commented Nov 15, 2021

kaovilai commented Nov 15, 2021

tremble left a comment

Choose a reason for hiding this comment

ansible-zuul bot left a comment

Choose a reason for hiding this comment