Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][connector-v2][FileSink] Allow the user to set the row delimiter as an empty string #3854

Merged
merged 1 commit into from
Jan 16, 2023

Conversation

lightzhao
Copy link
Contributor

If a user's text line is marked with a line delimiter, it is unnecessary to set the line delimiter, so set the line separator as an empty string

Purpose of this pull request

Check list

@lightzhao
Copy link
Contributor Author

@Hisoka-X @TyrantLucifer please review ,thanks.

Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lightzhao
Copy link
Contributor Author

@TaoZex please review, thanks.

@lightzhao
Copy link
Contributor Author

@Hisoka-X please review, thanks.

@Hisoka-X Hisoka-X requested a review from TyrantLucifer January 9, 2023 09:28
@lightzhao
Copy link
Contributor Author

@TyrantLucifer PTAL, thanks.

1 similar comment
@lightzhao
Copy link
Contributor Author

@TyrantLucifer PTAL, thanks.

@TyrantLucifer
Copy link
Member

IMO, This can lead to data disruption and lost data, when user sets a empty delimiter, all data will be wrote on one line. How can we read this type file in SeaTunnel? There may be 10 rows of data before writing and only 1 row after reading. cc @EricJoy2048 @CalvinKirs Hey guys, what do you think about it?

@lightzhao
Copy link
Contributor Author

IMO, This can lead to data disruption and lost data, when user sets a empty delimiter, all data will be wrote on one line. How can we read this type file in SeaTunnel? There may be 10 rows of data before writing and only 1 row after reading. cc @EricJoy2048 @CalvinKirs Hey guys, what do you think about it?

This case is aimed at the fact that the user data already contains a newline symbol, which allows the user to set an empty action.

@lightzhao
Copy link
Contributor Author

@EricJoy2048 @CalvinKirs PTAL.

@TaoZex
Copy link
Contributor

TaoZex commented Jan 16, 2023

Please incorporate the latest code changes in order to pass CI.

@TyrantLucifer TyrantLucifer merged commit 84508fc into apache:dev Jan 16, 2023
harveyyue pushed a commit to harveyyue/seatunnel that referenced this pull request Feb 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants