Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [File Connector V2] Data file name will duplicate when use SeaTunnel Engine #3611

Closed
2 of 3 tasks
EricJoy2048 opened this issue Nov 30, 2022 · 4 comments · Fixed by #3717
Closed
2 of 3 tasks
Assignees
Milestone

Comments

@EricJoy2048
Copy link
Member

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

image

image

SeaTunnel Version

ALL

SeaTunnel Config

CI: https://github.com/apache/incubator-seatunnel/pull/3456#pullrequestreview-1198820624

Running Command

CI: https://github.com/apache/incubator-seatunnel/pull/3456#pullrequestreview-1198820624

Error Exception

CI: https://github.com/apache/incubator-seatunnel/pull/3456#pullrequestreview-1198820624

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@TaoZex
Copy link
Contributor

TaoZex commented Nov 30, 2022

I want to try this issue.Please assign to me.

@TaoZex
Copy link
Contributor

TaoZex commented Dec 9, 2022

JobId was initialized before dag was resolved, but PipelineId was created for executeplan after dag was resolved. Therefore,
JobId and Pipelineid were created at different locations. I have communicated with @Hisoka-X ,this does not seem to be able to pass jobId+pipelneId directly to the connector.

@EricJoy2048
Copy link
Member Author

JobId was initialized before dag was resolved, but PipelineId was created for executeplan after dag was resolved. Therefore, JobId and Pipelineid were created at different locations. I have communicated with @Hisoka-X ,this does not seem to be able to pass jobId+pipelneId directly to the connector.

Yes, you are right. We plan fix it in file sink connector. @Hisoka-X will finish this work.

Thanks.

@TaoZex
Copy link
Contributor

TaoZex commented Dec 9, 2022

OK. Thanks for your reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
3 participants