Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [seatunnel-formats] Allow the entry in the map to be null and allow the key in the entry to be null #5277

Merged
merged 7 commits into from
Sep 11, 2023

Conversation

NickCodeJourney
Copy link
Contributor

@NickCodeJourney NickCodeJourney commented Aug 11, 2023

Allow the entry in the map to be null and allow the key in the entry to be null

Purpose of this pull request

Check list

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."
@NickCodeJourney
Copy link
Contributor Author

NickCodeJourney commented Aug 11, 2023

企业微信截图_16872569684521
When the filed value of map type column is null in hive,array out of bounds

yangpeng added 3 commits August 11, 2023 16:49
Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.
Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?
@NickCodeJourney NickCodeJourney changed the title [Bug] [seatunnel-formats] Add null check [Bug] [seatunnel-formats] Allow the entry in the map to be null and allow the key in the entry to be null Aug 11, 2023
@Hisoka-X Hisoka-X added the First-time contributor First-time contributor label Aug 26, 2023
@Hisoka-X Hisoka-X added this to the 2.3.4 milestone Aug 26, 2023
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! cc @TyrantLucifer

@EricJoy2048 EricJoy2048 merged commit 07105a5 into apache:dev Sep 11, 2023
@NickCodeJourney NickCodeJourney deleted the devfixhivemap branch September 11, 2023 10:17
Zhouwen-CN pushed a commit to Zhouwen-CN/seatunnel that referenced this pull request Sep 11, 2023
…llow the key in the entry to be null (apache#5277)

* 🐛 "Add null check in text deserialization

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."

* 🐛 Refactor text deserialization, allow null value assignments

Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.

* 🎨 Refactor Builder constructor in TextDeserializationSchema

Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?

* ✅ Add  test case where the value is empty and the key is empty in the map

---------

Co-authored-by: yangpeng <[email protected]>
Zhouwen-CN pushed a commit to Zhouwen-CN/seatunnel that referenced this pull request Sep 11, 2023
…llow the key in the entry to be null (apache#5277)

* 🐛 "Add null check in text deserialization

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."

* 🐛 Refactor text deserialization, allow null value assignments

Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.

* 🎨 Refactor Builder constructor in TextDeserializationSchema

Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?

* ✅ Add  test case where the value is empty and the key is empty in the map

---------

Co-authored-by: yangpeng <[email protected]>
Zhouwen-CN pushed a commit to Zhouwen-CN/seatunnel that referenced this pull request Sep 11, 2023
…llow the key in the entry to be null (apache#5277)

* 🐛 "Add null check in text deserialization

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."

* 🐛 Refactor text deserialization, allow null value assignments

Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.

* 🎨 Refactor Builder constructor in TextDeserializationSchema

Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?

* ✅ Add  test case where the value is empty and the key is empty in the map

---------

Co-authored-by: yangpeng <[email protected]>
gnehil pushed a commit to gnehil/seatunnel that referenced this pull request Oct 12, 2023
…llow the key in the entry to be null (apache#5277)

* 🐛 "Add null check in text deserialization

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."

* 🐛 Refactor text deserialization, allow null value assignments

Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.

* 🎨 Refactor Builder constructor in TextDeserializationSchema

Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?

* ✅ Add  test case where the value is empty and the key is empty in the map

---------

Co-authored-by: yangpeng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants