Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamodb2: ProjectionExpression with ExpressionAttributeNames breaks with prefixes #2189

Closed
beheh opened this issue May 8, 2019 · 4 comments

Comments

@beheh
Copy link

beheh commented May 8, 2019

I'm trying to query a table using moto's dynamodb2 backend on moto==1.3.7. Specifically, I'm querying my table with the following parameters:

{
  "ProjectionExpression": "#1, #2, #3, #4, #5, #6, #7, #8, #9, #10, #11, #12, #13, #14, #15",
  "ExpressionAttributeNames": {"#0": "foo", "#1": "bar", "#2": "baz", "#3": "baf", "#4": "puf", "#5": "par", "#6": "paz", "#7": "pauz", "#8": "pop", "#9": "plop", "#10": "plip", "#11": "plap", "#12": "plup", "#13": "bla", "#14": "ble", "#15": "bli"},
  ...
}

Note how #1 is a prefix of #10, #11, #12, #13, #14 and #15.

I'd expect moto to treat this like the following ProjectionExpression:

"foo, bar, baz, baf, puf, par, paz, pauz, pop, plop, plip, plap, plup, bla, ble, bli"

What happens instead:

"foo, bar, baz, baf, puf, par, paz, pauz, pop, plop, plip, foo1, foo2, foo3, foo4, foo5"

This is due to the following code in moto/dynamodb2/responses.py:

https://github.com/spulec/moto/blob/8cb4db1896521af99ff295b028f604c6ab599401/moto/dynamodb2/responses.py#L405-L409

Note how projection_expression.replace does a single string replacement, which means #1 will match the first two characters of #10, #11 and so on. A more graceful variant of that code would probably use the expressions after splitting to replace them one by one and reassembling to reassemble projection_expression.

All in all this results in an incomplete response missing the attributes with the AttributeName #10 and above. I've tested this against AWS and the error does not occur there.

I ran over this when using pynamodb, as it automatically creates this numerically indexed ProjectionExpression based on a list of attributes to get.

@beheh beheh changed the title dynamodb2: ProjectionExpression with ExpressionAttributeNames dynamodb2: ProjectionExpression with ExpressionAttributeNames breaks with prefixes May 8, 2019
@beheh
Copy link
Author

beheh commented May 8, 2019

A very quick improvement of the bugged snippet from moto above might look like this:

        if projection_expression and expression_attribute_names:
            expressions = [x.strip() for x in projection_expression.split(',')]
            projection_expression = None
            for expression in expressions:
                if projection_expression is not None:
                    projection_expression = projection_expression + ", "
                else:
                    projection_expression = ""
                if expression in expression_attribute_names:
                    projection_expression = projection_expression + expression_attribute_names[expression]
                else:
                    projection_expression = projection_expression + expression

@spulec
Copy link
Collaborator

spulec commented Jul 9, 2019

Thanks. A pull request would be very welcome!

@joolean
Copy link
Contributor

joolean commented Sep 9, 2019

As requested! #2412

@spulec
Copy link
Collaborator

spulec commented Sep 12, 2019

Closed with #2412

@spulec spulec closed this as completed Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants