Error creating GSI with global_keys_only #967

nick-zh · 2022-03-06T15:57:16Z

Summary

I get an error when i create a GSI for dynamodb with global_keys_only
From what i understand i don't need to include any attributes, but somehow creation gives an error

Issue Type

Bug Report

Component Name

dynamodb_table

Ansible Version

$ ansible --version

ansible 2.10.8
  config file = ansible/ansible.cfg
  configured module search path = ['.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.9.10 (main, Feb 22 2022, 13:54:07) [GCC 11.2.0]

Collection Versions

$ ansible-galaxy collection list

Collection                Version
------------------------- -------
amazon.aws                1.4.0  
ansible.netcommon         1.5.0  
ansible.posix             1.1.1  
ansible.windows           1.4.0  
arista.eos                1.3.0  
awx.awx                   14.1.0 
azure.azcollection        1.4.0  
check_point.mgmt          1.0.6  
chocolatey.chocolatey     1.0.2  
cisco.aci                 1.1.1  
cisco.asa                 1.0.4  
cisco.intersight          1.0.10 
cisco.ios                 1.3.0  
cisco.iosxr               1.2.1  
cisco.meraki              2.2.0  
cisco.mso                 1.1.0  
cisco.nso                 1.0.3  
cisco.nxos                1.4.0  
cisco.ucs                 1.6.0  
cloudscale_ch.cloud       1.3.1  
community.aws             1.3.0  
community.azure           1.0.0  
community.crypto          1.4.0  
community.digitalocean    1.0.0  
community.docker          1.2.2  
community.fortios         1.0.0  
community.general         1.3.6  
community.google          1.0.0  
community.grafana         1.1.0  
community.hashi_vault     1.1.0  
community.hrobot          1.1.0  
community.kubernetes      1.1.1  
community.kubevirt        1.0.0  
community.libvirt         1.0.0  
community.mongodb         1.2.0  
community.mysql           1.2.0  
community.network         1.3.2  
community.okd             1.0.0  
community.postgresql      1.1.1  
community.proxysql        1.0.0  
community.rabbitmq        1.0.1  
community.routeros        1.1.0  
community.skydive         1.0.0  
community.vmware          1.7.0  
community.windows         1.3.0  
community.zabbix          1.2.0  
containers.podman         1.4.1  
cyberark.conjur           1.1.0  
cyberark.pas              1.0.5  
dellemc.os10              1.0.2  
dellemc.os6               1.0.6  
dellemc.os9               1.0.3  
f5networks.f5_modules     1.7.1  
fortinet.fortimanager     1.0.5  
fortinet.fortios          1.1.8  
frr.frr                   1.0.3  
gluster.gluster           1.0.1  
google.cloud              1.0.2  
hetzner.hcloud            1.2.1  
ibm.qradar                1.0.3  
infinidat.infinibox       1.2.4  
junipernetworks.junos     1.3.0  
mellanox.onyx             1.0.0  
netapp.aws                20.9.0 
netapp.elementsw          20.11.0
netapp.ontap              20.12.0
netapp_eseries.santricity 1.1.0  
netbox.netbox             1.2.1  
ngine_io.cloudstack       1.2.0  
ngine_io.exoscale         1.0.0  
ngine_io.vultr            1.1.0  
openstack.cloud           1.2.1  
openvswitch.openvswitch   1.1.0  
ovirt.ovirt               1.3.0  
purestorage.flasharray    1.6.2  
purestorage.flashblade    1.4.0  
servicenow.servicenow     1.0.4  
splunk.es                 1.0.2  
theforeman.foreman        1.5.1  
vyos.vyos                 1.1.1  
wti.remote                1.0.1  

# .ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    3.0.0  
community.aws 2.1.0

AWS SDK versions

$ pip show boto boto3 botocore

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.9/dist-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.18.47
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: .local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.21.47
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: .local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

- name: Create dynamo table with hash and range primary key
  community.aws.dynamodb_table:
    name: MyTable
    region: eu-west-1
    hash_key_name: PrimaryId
    hash_key_type: STRING
    range_key_name: SecondaryId
    range_key_type: STRING
    billing_mode: PAY_PER_REQUEST
    indexes:
      - name: SecondaryIndex
        type: global_keys_only
        hash_key_name: SecondaryId

Expected Results

Table is created and i only need to add to add the non key attributes if use the modes global_include and include as stated here

Actual Results

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Invalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1
fatal: [localhost]: FAILED! => {"boto3_version": "1.18.47", "botocore_version": "1.21.47", "changed": false, "msg": "Failed to update table: Parameter validation failed:\nInvalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1"}

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2022-03-06T16:01:39Z

Files identified in the description:

[plugins/modules/dynamodb_table.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/dynamodb_table.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2022-03-06T16:01:40Z

cc @jillr @loia @markuman @s-hertel @tremble
click here for bot help

Sonic0 · 2022-05-25T12:53:47Z

I have the same issue with Ansible 5.8.0, ansible-core 2.12.5, collection community.aws 2.4 and Python 3.8.10

tremble · 2022-05-25T14:04:21Z

@Sonic0 Is there any chance you could create your intended DynamoDB table and provide the output of aws dynamodb describe-table --table-name <table_name>

I'm not particularly familiar with DynamoDB, but might be able to fix the issue if you're able to describe the end state.

nick-zh · 2022-05-25T14:44:11Z

@tremble i have done that on an example table (removed region and arn), i hope this is what you were looking for

{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "ProfileId",
                "AttributeType": "S"
            },
            {
                "AttributeName": "UserId",
                "AttributeType": "S"
            }
        ],
        "TableName": "TestTable",
        "KeySchema": [
            {
                "AttributeName": "UserId",
                "KeyType": "HASH"
            },
            {
                "AttributeName": "ProfileId",
                "KeyType": "RANGE"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": "2022-05-25T16:40:34.413000+02:00",
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0,
            "ReadCapacityUnits": 5,
            "WriteCapacityUnits": 5
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable",
        "TableId": "<redacted>",
        "GlobalSecondaryIndexes": [
            {
                "IndexName": "ProfileId-index",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "KEYS_ONLY"
                },
                "IndexStatus": "CREATING",
                "Backfilling": true,
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index"
            }
        ]
    }
}

tremble · 2022-05-25T14:46:04Z

@nick-zh Yup, that's what I needed.

nick-zh · 2022-05-25T15:00:14Z

Here are what the other index types look like, maybe that's helpful as well:

{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "ProfileId",
                "AttributeType": "S"
            },
            {
                "AttributeName": "UserId",
                "AttributeType": "S"
            }
        ],
        "TableName": "TestTable",
        "KeySchema": [
            {
                "AttributeName": "UserId",
                "KeyType": "HASH"
            },
            {
                "AttributeName": "ProfileId",
                "KeyType": "RANGE"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": "2022-05-25T16:40:34.413000+02:00",
        "ProvisionedThroughput": {
            "LastDecreaseDateTime": "2022-05-25T16:53:38.830000+02:00",
            "NumberOfDecreasesToday": 2,
            "ReadCapacityUnits": 1,
            "WriteCapacityUnits": 1
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable",
        "TableId": "<redacted>",
        "GlobalSecondaryIndexes": [
            {
                "IndexName": "ProfileId-index",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "ALL"
                },
                "IndexStatus": "ACTIVE",
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index"
            },
            {
                "IndexName": "ProfileId-index2",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "INCLUDE",
                    "NonKeyAttributes": [
                        "UserId"
                    ]
                },
                "IndexStatus": "ACTIVE",
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index2"
            }
        ]
    }
}

tremble · 2022-05-25T15:34:27Z

@nick-zh Any chance you could try #1162 ?

nick-zh · 2022-05-25T16:24:45Z

I was not sure what's the best way to do this, but i adjusted the fix of plugins/modules/dynamodb_table.py to my local collection and i was able to successfully create a table with global_keys_only 🎉
Many thanks @tremble 🙏 (as always)

Sonic0 · 2022-05-25T16:35:16Z

I applied the fix to the file plugins/modules/dynamodb_table.py in my local collection and now it is working!!!
Many thanks @tremble! 🥳 🥳

Fix issue when creating GSI with global_keys_only SUMMARY fixes: #967 Undocumented requirement on NonKeyAttributes that it should be omitted rather than an empty list. ISSUE TYPE Bugfix Pull Request COMPONENT NAME dynamodb_table ADDITIONAL INFORMATION An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Invalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1 fatal: [localhost]: FAILED! => {"boto3_version": "1.18.47", "botocore_version": "1.21.47", "changed": false, "msg": "Failed to update table: Parameter validation failed:\nInvalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1"} Reviewed-by: Markus Bergholz <[email protected]>

Fix issue when creating GSI with global_keys_only SUMMARY fixes: #967 Undocumented requirement on NonKeyAttributes that it should be omitted rather than an empty list. ISSUE TYPE Bugfix Pull Request COMPONENT NAME dynamodb_table ADDITIONAL INFORMATION An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Invalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1 fatal: [localhost]: FAILED! => {"boto3_version": "1.18.47", "botocore_version": "1.21.47", "changed": false, "msg": "Failed to update table: Parameter validation failed:\nInvalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1"} Reviewed-by: Markus Bergholz <[email protected]> (cherry picked from commit 690e250)

[PR #1162/690e250a backport][stable-3] Fix issue when creating GSI with global_keys_only This is a backport of PR #1162 as merged into main (690e250). SUMMARY fixes: #967 Undocumented requirement on NonKeyAttributes that it should be omitted rather than an empty list. ISSUE TYPE Bugfix Pull Request COMPONENT NAME dynamodb_table ADDITIONAL INFORMATION An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Invalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1 fatal: [localhost]: FAILED! => {"boto3_version": "1.18.47", "botocore_version": "1.21.47", "changed": false, "msg": "Failed to update table: Parameter validation failed:\nInvalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1"} Reviewed-by: Markus Bergholz <[email protected]>

… (ansible-collections#967) fix ignore_nonexistent_bucket bug for listing (ansible-collections#966) SUMMARY remove duplicated use of bucket_check() and reuse bucketrtn instead Fixes ansible-collections#966 ISSUE TYPE Bugfix Pull Request COMPONENT NAME s3_object ADDITIONAL INFORMATION Reviewed-by: Mark Chappell <None>

ansibullbot added bug This issue/PR relates to a bug module module needs_triage plugins plugin (any type) python3 traceback labels Mar 6, 2022

ansibullbot removed the needs_triage label May 25, 2022

tremble mentioned this issue May 25, 2022

Fix issue when creating GSI with global_keys_only #1162

Merged

ansibullbot added the has_pr label May 25, 2022

softwarefactory-project-zuul bot closed this as completed in #1162 May 26, 2022

patchback bot mentioned this issue May 26, 2022

[PR #1162/690e250a backport][stable-3] Fix issue when creating GSI with global_keys_only #1169

Merged

renovate bot mentioned this issue May 30, 2022

chore(deps): update dependency community.aws to v3.3.0 linxside/ansible-ee#24

Merged

1 task

renovate bot mentioned this issue Jun 24, 2022

chore(deps): update dependency community.aws to v4 linxside/ansible-ee#34

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error creating GSI with global_keys_only #967

Error creating GSI with global_keys_only #967

nick-zh commented Mar 6, 2022 •

edited

Loading

ansibullbot commented Mar 6, 2022

ansibullbot commented Mar 6, 2022

Sonic0 commented May 25, 2022 •

edited

Loading

tremble commented May 25, 2022

nick-zh commented May 25, 2022 •

edited

Loading

tremble commented May 25, 2022

nick-zh commented May 25, 2022

tremble commented May 25, 2022

nick-zh commented May 25, 2022 •

edited

Loading

Sonic0 commented May 25, 2022

Error creating GSI with global_keys_only #967

Error creating GSI with global_keys_only #967

Comments

nick-zh commented Mar 6, 2022 • edited Loading

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

ansibullbot commented Mar 6, 2022

ansibullbot commented Mar 6, 2022

Sonic0 commented May 25, 2022 • edited Loading

tremble commented May 25, 2022

nick-zh commented May 25, 2022 • edited Loading

tremble commented May 25, 2022

nick-zh commented May 25, 2022

tremble commented May 25, 2022

nick-zh commented May 25, 2022 • edited Loading

Sonic0 commented May 25, 2022

nick-zh commented Mar 6, 2022 •

edited

Loading

Sonic0 commented May 25, 2022 •

edited

Loading

nick-zh commented May 25, 2022 •

edited

Loading

nick-zh commented May 25, 2022 •

edited

Loading