Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ak.merge_union_of_records #2185

Merged
merged 18 commits into from
Feb 2, 2023

Conversation

agoose77
Copy link
Collaborator

@agoose77 agoose77 commented Feb 1, 2023

This PR addresses #2182. We already implement some of this machinery in UnionArray, which has a _union_of_optionarrays method to convert from option[union[X, Y, Z]] to union[option[X], Y, ..., Z]. It doesn't seem like that is particularly useful in this case.

Interestingly there is some overlap with ak._util.union_to_record.

Added functions

  • ak.merge_option_of_records
  • ak.merge_union_of_records

@codecov
Copy link

codecov bot commented Feb 1, 2023

Codecov Report

Merging #2185 (f3ff3b3) into main (a91d96b) will increase coverage by 0.08%.
The diff coverage is 88.76%.

Additional details and impacted files
Impacted Files Coverage Δ
...c/awkward/operations/ak_merge_option_of_records.py 88.00% <88.00%> (ø)
...rc/awkward/operations/ak_merge_union_of_records.py 88.70% <88.70%> (ø)
src/awkward/operations/__init__.py 100.00% <100.00%> (ø)
src/awkward/operations/ak_where.py 91.83% <0.00%> (ø)
src/awkward/operations/ak_fill_none.py 95.12% <0.00%> (ø)
src/awkward/operations/ak_full_like.py 98.18% <0.00%> (ø)
src/awkward/operations/ak_to_backend.py 100.00% <0.00%> (ø)
src/awkward/operations/ak_with_field.py 100.00% <0.00%> (ø)
src/awkward/contents/recordarray.py 84.91% <0.00%> (+0.41%) ⬆️
src/awkward/index.py 90.44% <0.00%> (+0.63%) ⬆️
... and 2 more

@pre-commit-ci pre-commit-ci bot temporarily deployed to docs-preview February 1, 2023 12:00 Inactive
def apply_displace_index(layout, backend, **kwargs):
if layout.is_record:
return layout
elif (layout.is_option or layout.is_indexed) and layout.content.is_record:
Copy link
Collaborator Author

@agoose77 agoose77 Feb 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is doing something different for is_option. merge_option_of_records

@agoose77 agoose77 requested a review from jpivarski February 1, 2023 14:44
@agoose77 agoose77 marked this pull request as ready for review February 1, 2023 14:44
@pre-commit-ci pre-commit-ci bot temporarily deployed to docs-preview February 1, 2023 16:53 Inactive
@agoose77
Copy link
Collaborator Author

agoose77 commented Feb 1, 2023

@jpivarski what I'm not certain on here is what should happen for esoteric layouts that have multiple "option-of-record", and "union-of-record" in a single dimension. What should we do here, first wins? It seems like we do that in ak.drop_none(), but I don't recall us having a rule.

@agoose77 agoose77 temporarily deployed to docs-preview February 1, 2023 17:44 — with GitHub Actions Inactive
@pre-commit-ci pre-commit-ci bot temporarily deployed to docs-preview February 1, 2023 20:14 Inactive
Copy link
Member

@jpivarski jpivarski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some tests which include the original problem in #2182 and a variation of it that has already-optional fields (to verify that the simplified classmethod is being used).

I also added tests for the ak.merge_option_of_records, and the result is wrong; it drops Nones, rather than merging them into the record fields. I'm labeling this review as "Approve" because once that's fixed and the tests pass, it will be ready to merge.

Oh, I didn't add any tests for different axis values. Was that straightforward/formulaic? If it follows a pattern set by the many other functions that use axis, then I don't think we need to explicitly test it.

Darn it, I just convinced myself that I really do need to test it. One moment...

Okay, tested it.

@jpivarski jpivarski temporarily deployed to docs-preview February 1, 2023 20:27 — with GitHub Actions Inactive
@agoose77 agoose77 temporarily deployed to docs-preview February 1, 2023 20:36 — with GitHub Actions Inactive
@agoose77 agoose77 temporarily deployed to docs-preview February 2, 2023 00:08 — with GitHub Actions Inactive
@agoose77 agoose77 enabled auto-merge (squash) February 2, 2023 00:19
@agoose77 agoose77 temporarily deployed to docs-preview February 2, 2023 00:26 — with GitHub Actions Inactive
@agoose77 agoose77 merged commit 4274552 into main Feb 2, 2023
@agoose77 agoose77 deleted the agoose77/feat-merge-union-of-records branch February 2, 2023 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants