Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle missing fields as nulls in get_json_object() #10970

Merged

Conversation

SrikarVanavasam
Copy link
Contributor

@SrikarVanavasam SrikarVanavasam commented May 25, 2022

Addresses: #10196

Previously, get_json_object() ignored fields in a JsonPath expression that are missing in the json string. This PR adds the option to return these missing fields as null instead.

@GPUtester
Copy link
Collaborator

Can one of the admins verify this patch?

@SrikarVanavasam SrikarVanavasam requested a review from a team as a code owner May 25, 2022 15:31
@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label May 25, 2022
@SrikarVanavasam SrikarVanavasam marked this pull request as draft May 25, 2022 15:32
@SrikarVanavasam SrikarVanavasam marked this pull request as ready for review May 25, 2022 15:35
@SrikarVanavasam SrikarVanavasam marked this pull request as draft May 25, 2022 15:35
@vyasr vyasr added non-breaking Non-breaking change bug Something isn't working 2 - In Progress Currently a work in progress labels May 25, 2022
@raydouglass
Copy link
Member

okay to test

@SrikarVanavasam SrikarVanavasam added the improvement Improvement / enhancement to an existing function label May 26, 2022
@SrikarVanavasam SrikarVanavasam removed the bug Something isn't working label May 26, 2022
@codecov
Copy link

codecov bot commented May 26, 2022

Codecov Report

❗ No coverage uploaded for pull request base (branch-22.08@aec9007). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head 34d8bfe differs from pull request most recent head 30ee5d9. Consider uploading reports for the commit 30ee5d9 to get more accurate results

@@               Coverage Diff               @@
##             branch-22.08   #10970   +/-   ##
===============================================
  Coverage                ?   86.34%           
===============================================
  Files                   ?      144           
  Lines                   ?    22738           
  Branches                ?        0           
===============================================
  Hits                    ?    19632           
  Misses                  ?     3106           
  Partials                ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aec9007...30ee5d9. Read the comment docs.

@nvdbaranec
Copy link
Contributor

@mlahir1 Do you guys have some time to try this out and see if it solves your issue?

@nvdbaranec
Copy link
Contributor

rerun tests

Copy link
Contributor

@ttnghia ttnghia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't see any tests for set_missing_fields_as_nulls = false.

@SrikarVanavasam
Copy link
Contributor Author

I also don't see any tests for set_missing_fields_as_nulls = false.

set_missing_fields_as_nulls = false is set by default so I figured the existing tests would cover that case. Should I also have cases that explicitly set the condition?

@ttnghia
Copy link
Contributor

ttnghia commented Jun 7, 2022

set_missing_fields_as_nulls = false is set by default so I figured the existing tests would cover that case. Should I also have cases that explicitly set the condition?

I think so. We don't know, and are not guanteed, if the existing tests have that covered or not.

@mythrocks
Copy link
Contributor

set_missing_fields_as_nulls = false is set by default so I figured the existing tests would cover that case. Should I also have cases that explicitly set the condition?

I think so. We don't know, and are not guanteed, if the existing tests have that covered or not.

Agreeing with @ttnghia's assessment. I meant to post this as well: It would be good to test set_missing_fields_as_nulls = true case alongside the false. It will underline the difference in behaviour.

@SrikarVanavasam SrikarVanavasam requested a review from ttnghia June 13, 2022 15:27
Co-authored-by: Nghia Truong <[email protected]>
@SrikarVanavasam SrikarVanavasam requested a review from ttnghia June 13, 2022 20:35
Copy link
Contributor

@mythrocks mythrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing the review comments. LGTM.

@SrikarVanavasam
Copy link
Contributor Author

rerun tests

@SrikarVanavasam
Copy link
Contributor Author

@gpucibot merge

1 similar comment
@nvdbaranec
Copy link
Contributor

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 40ec190 into rapidsai:branch-22.08 Jun 21, 2022
@mlahir1
Copy link

mlahir1 commented Jun 21, 2022

@mlahir1 Do you guys have some time to try this out and see if it solves your issue?

Thank you very much for the solving this. Sure, i belv this is merged in 22.08 nightly. i will test this sometime this week and check of this solves our issue.

@mlahir1
Copy link

mlahir1 commented Jun 21, 2022

@SrikarVanavasam , I could find any api changes on python side. Are the missing automatically assigned as Nulls or do i need to pass in some flag in the python API side.

@nvdbaranec
Copy link
Contributor

Hi @mlahir1 . Yeah, sorry we missed this. We'll get a PR up with it soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants