Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: preserve list meanings #575

Merged
merged 13 commits into from
Dec 12, 2024
Merged

fix: preserve list meanings #575

merged 13 commits into from
Dec 12, 2024

Conversation

daniel-sanche
Copy link
Contributor

@daniel-sanche daniel-sanche commented Dec 5, 2024

Currently, the library only keeps track of the meaning field of sub-elements of array_values it sees. We also need it to keep track of the meaning of the array itself

This PR addresses this by returning a tuple of (root_meaning, sub_meaning_list) instead of just the sub_meaning_list in _get_meaning(), and using the tuple to populate the fields properly in _set_pb_meaning_from_entity()

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: datastore Issues related to the googleapis/python-datastore API. labels Dec 5, 2024
@daniel-sanche daniel-sanche changed the title [DRAFT] chore: preserve list meanings chore: preserve list meanings Dec 6, 2024
@daniel-sanche daniel-sanche marked this pull request as ready for review December 6, 2024 21:04
@daniel-sanche daniel-sanche requested review from a team as code owners December 6, 2024 21:04
@bhshkh bhshkh assigned daniel-sanche and unassigned bhshkh Dec 9, 2024

return None
sub_meanings = [sub_value_pb.meaning or None for sub_value_pb in values]
if not any(meaning is not None for meaning in sub_meanings):
Copy link

@NickChittle NickChittle Dec 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, why wouldn't we just return a tuple of (None, []), instead of transforming the return type here to just a single None?

(To me it would be simpler to have a guaranteed format for List, and a guaranteed format for non-lists, or maybe this saves some RAM?)

Copy link
Contributor Author

@daniel-sanche daniel-sanche Dec 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case it's not clear: the reason for this transformation is to flatten giant lists of [None, None, None, ...], so we don't have to iterate over the whole thing again later, when there are no meanings present (which is probably the most common situation)

You're right that we could return an empty list instead of None here to signal that there are no meanings present. But it's a trade-off. That would buy us type consistency, but then we'd lose the property that the sub-meaning list is always be the same size as the original array. We'd need to remember to do size comparisons before iterating, so we don't really win anything from that change. And size mismatches wouldn't be flagged by type checkers like missing None checks, so those bugs would be harder to notice.

TL;DR: This is a special case in our processing logic, and it's usually safer to use None to signal special cases, instead of returning lists with unpredictable sizes

@daniel-sanche daniel-sanche changed the title chore: preserve list meanings fix: preserve list meanings Dec 12, 2024
@daniel-sanche daniel-sanche merged commit 266243b into main Dec 12, 2024
29 checks passed
@daniel-sanche daniel-sanche deleted the preserve_list_meanings branch December 12, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the googleapis/python-datastore API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants