Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [formats] Fix fail to parse line when content contains the file delimiter #6589

Merged
merged 19 commits into from
Apr 8, 2024

Conversation

litiliu
Copy link
Contributor

@litiliu litiliu commented Mar 27, 2024

Purpose of this pull request

This close #6588
This PR import the apache-common-csv to parse the line of csv file to ensure the csv file's line will not be split if it contains quotation marks.
if the original content is

a,"b,c",d

and the delimiter is "," then it will be split into

a     b,c    d

instead of

a "b  c"  d

Does this PR introduce any user-facing change?

No

How was this patch tested?

unit test has been added for the above case.

Check list

@HsbcJone
Copy link
Contributor

auto failed

@EricJoy2048
Copy link
Member

Please fix ci error.

@Hisoka-X Hisoka-X added the bug label Mar 28, 2024
@hailin0 hailin0 requested a review from TyrantLucifer March 30, 2024 14:14
@litiliu litiliu requested a review from TyrantLucifer April 3, 2024 03:39
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hailin0 hailin0 merged commit 17e2918 into apache:dev Apr 8, 2024
5 checks passed
@litiliu litiliu deleted the fix-6588 branch April 8, 2024 15:07
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] [Seatunnel-formats] Fail to parse line if content contains the file delimiter
6 participants