Route53 boto error "Profile not found" when using IAM Role with AWS config

[PierreBeucher]

Migrated from ansible/ansible#68711

SUMMARY

When using route53 module using Assumed Role based authentication, module will fail with an error like:

The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.cloud.amazon.route53', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/local/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py", line 701, in <module>
  File "/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py", line 595, in main
  File "/usr/local/lib/python3.7/site-packages/boto/route53/connection.py", line 88, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/connection.py", line 555, in __init__
    profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/provider.py", line 201, in __init__
    self.get_credentials(access_key, secret_key, security_token, profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/provider.py", line 297, in get_credentials
    profile_name)
boto.provider.ProfileNotFoundError: Profile "my-profile" not found!
fatal: [127.0.0.1]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.cloud.amazon.route53', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py\", line 701, in <module>\n  File \"/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py\", line 595, in main\n  File \"/usr/local/lib/python3.7/site-packages/boto/route53/connection.py\", line 88, in __init__\n    profile_name=profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/connection.py\", line 555, in __init__\n    profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/provider.py\", line 201, in __init__\n    self.get_credentials(access_key, secret_key, security_token, profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/provider.py\", line 297, in get_credentials\n    profile_name)\nboto.provider.ProfileNotFoundError: Profile \"my-profile\" not found!\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

May be related to ansible/ansible#41185, but this is a Bug not a Feature Request as this method of authentication with Boto is available and works fine with other modules.

ISSUE TYPE

• Bug Report

COMPONENT NAME

route53 module

ANSIBLE VERSION

ansible 2.9.6
  config file = None
  configured module search path = ['/home/gitops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.4 (default, Aug 21 2019, 00:19:59) [GCC 8.3.0]

CONFIGURATION

DEFAULT_HOST_LIST(env: ANSIBLE_INVENTORY) = ['/gitops/inventories/infra-dev']
DEFAULT_VAULT_PASSWORD_FILE(env: ANSIBLE_VAULT_PASSWORD_FILE) = /gitops/.vault/infra-dev

OS / ENVIRONMENT

$ cat /etc/*release
3.10.2
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.10.2
PRETTY_NAME="Alpine Linux v3.10"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"

STEPS TO REPRODUCE

Using AWS config defining a profile route53-role-profile assuming a Role such as:

# content of ~/.aws/config
[profile route53-source-profile]
region = eu-central-1

[profile route53-role-profile]
region = eu-central-1
role_arn = arn:aws:iam::12345678910:role/Route53Role
source_profile = route53-source-profile

# content of ~/.aws/credentials
[route53-source-profile]
aws_access_key_id = XXXX
aws_secret_access_key = secret

With task such as:

# Use profile assuming our Role
# Cause mentionned bug
- route53:
    state: present
    profile: route53-role-profile
    hosted_zone_id: "my.zone.ai"
    record: "*.my.zone.ai"
    type: CNAME
    value: "0.0.0.0"

Will cause mentionned error.

Same result when using AWS_PROFILE environment variable instead of profile:

But using the profile on which access keys are configured directly will work:

# Works fine
- route53:
    state: present
    profile: route53-source-profile
    hosted_zone_id: "my.zone.ai"
    record: "*.my.zone.ai"
    type: CNAME
    value: "0.0.0.0"

Using AWS CLI to perform similar actions with such config works fine.

EXPECTED RESULTS

route53 module to use boto and properly assume configured role to execute task.

ACTUAL RESULTS

Module fail with error:

The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.cloud.amazon.route53', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/local/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py", line 701, in <module>
  File "/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py", line 595, in main
  File "/usr/local/lib/python3.7/site-packages/boto/route53/connection.py", line 88, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/connection.py", line 555, in __init__
    profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/provider.py", line 201, in __init__
    self.get_credentials(access_key, secret_key, security_token, profile_name)
  File "/usr/local/lib/python3.7/site-packages/boto/provider.py", line 297, in get_credentials
    profile_name)
boto.provider.ProfileNotFoundError: Profile "my-profile" not found!
fatal: [127.0.0.1]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1586184741.8429592-23677387734274/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.cloud.amazon.route53', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/local/lib/python3.7/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py\", line 701, in <module>\n  File \"/tmp/ansible_route53_payload_55fkdb4s/ansible_route53_payload.zip/ansible/modules/cloud/amazon/route53.py\", line 595, in main\n  File \"/usr/local/lib/python3.7/site-packages/boto/route53/connection.py\", line 88, in __init__\n    profile_name=profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/connection.py\", line 555, in __init__\n    profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/provider.py\", line 201, in __init__\n    self.get_credentials(access_key, secret_key, security_token, profile_name)\n  File \"/usr/local/lib/python3.7/site-packages/boto/provider.py\", line 297, in get_credentials\n    profile_name)\nboto.provider.ProfileNotFoundError: Profile \"my-profile\" not found!\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

[Riice]

Any feedback here? We initially tried to work around this, but have removed the module and replaced it with aws cli calls for the time being.

[steffakasid]

Same issue here. Would be nice to have it solved. Also switched back to aws cli to create RessourceRecords.

[dhikrahashim]

Any fix for this. I am able to use profile and able to run the command. But the same playbook is not working in Ansible Tower. I used AWS creds and profile in Tower it is not working.

Tower dynamic inventory is working using iam role arn

[tremble]

CC @bpennypacker and @jimbydamonk (the listed authors of this module)

I apologise for the delayed response.

This module is specifically an older boto v2 based module rather than a boto v3 based module.

This means that the configuration of the module isn't taken from the 'usual' places and doesn't have some of the features you might want (including automatically assuming an IAM Role). See also: http://boto.cloudhackers.com/en/latest/boto_config_tut.html

The long term fix would be to migrate this module to boto3 and patches would be welcome, we already have some automated testing in place which would help when it comes to reviewing the change. As a work around you could use the 'source' profile to assume the role with the sts_assume_role module, and then pass the generated credentials in as parameters.

[ansibullbot]

cc @jillr @s-hertel @wimnat
click here for bot help

[eRadical]

This will be covered by pull request #405.

[tremble]

#405 was merged. This issue should be fixed in the latest versions of the module.

[tremble]

To copy from the comment on #405 for anyone else who might find this.

(ansible-dev) [17:21:16+0100] ~/vcs/ansible 
[✔ ansible] $ ansible-playbook test.yml 
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current version: 3.6.8 (default, Aug 18 2020, 08:33:21) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]. This
 feature will be removed from ansible-core in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: You are running the development version of Ansible. You should only run Ansible from "devel" if you are modifying the Ansible engine, or trying out features under development. This is a rapidly
changing source of code and can become unstable at any point.
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [community.aws.route53] **************************************************************************************************************************************************************************************
changed: [localhost]

TASK [community.aws.route53] **************************************************************************************************************************************************************************************
changed: [localhost]

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

(ansible-dev) [17:21:23+0100] ~/vcs/ansible 
[✔ ansible] $ cat test.yml 
- hosts: localhost
  gather_facts: False
  connection: local
  collections:
  - amazon.aws
  - community.aws
  tasks:
  - community.aws.route53:
      zone: example.test
      record: home-sweet-home.example.test
      type: A
      value: 127.0.0.1
      state: present
      profile: myprofile
  - community.aws.route53:
      zone: example.test
      record: home-sweet-home.example.test
      type: A
      value: 127.0.0.1
      state: absent
      profile: myprofile

One key piece is that "profile" is passed directly to the boto3 library, if you don't use "local", then you'll need to make sure that the profile is available wherever the module is actually being executed (note: this will be the Ansible target not the controller.)

@eRadical also said:

Yup... my bad.

I was inheriting become: true from the playbook/role.
Needless to say that on my machine there is no AWS profile under root.