Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FlinkX on yarn per-job模式运行的最终状态错误 #258

Closed
AtomMarker opened this issue Jul 29, 2020 · 2 comments
Closed

FlinkX on yarn per-job模式运行的最终状态错误 #258

AtomMarker opened this issue Jul 29, 2020 · 2 comments

Comments

@AtomMarker
Copy link

在FlinkX进行传输的过程中产生错误,比如在mysqlwriter中配置一个错误的url,这样的异常是会在工具内部捕获。
因此,在per-job模式下,yarn的会话是感知不到你内部产生了异常的。

在com.dtstack.flinkx.Main::main() 函数中,并没有做result的其他处理。

JobExecutionResult result = env.execute(jobIdString);
        if(env instanceof MyLocalStreamEnvironment){
            ResultPrintUtil.printResult(result);
        }

这样产生的效果就是,本来应该提示错误的一次传输,在调度系统通过yarn查看状态的时候就变成了成功,导致后面的流程非法运行。

@AtomMarker
Copy link
Author

这个问题跟Flink的yarn模式有关,可能得从Flink的源码入手。
这里有一个Spark退出时反馈给Yarn状态的修复方式,可以参考

apache/spark#2577

我再研究一下Flink的源码和Issues

@FlechazoW
Copy link
Member

This issue has been too long to review it and reopen new one if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants