Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hotfix] Fix redis sink NPE #8171

Merged
merged 5 commits into from
Dec 2, 2024
Merged

[Hotfix] Fix redis sink NPE #8171

merged 5 commits into from
Dec 2, 2024

Conversation

lm-ylj
Copy link
Contributor

@lm-ylj lm-ylj commented Nov 29, 2024

Purpose of this pull request

Fix redis sink NPE when upstream data is null,exception:
2024-11-28 11:16:08,615 INFO [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.generic-operation.thread-22] - Job SeaTunnel_Job (914353897047326721), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-jdbc]-SourceTask (1/1)] state process is stopped
2024-11-28 11:16:08,615 ERROR [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.generic-operation.thread-22] - Job SeaTunnel_Job (914353897047326721), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-jdbc]-SourceTask (1/1)] end with state FAILED and Exception: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:292)
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:74)
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39)
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27)
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75)
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50)
at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51)
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:721)
at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1043)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.subSinkErrorCheck(MultiTableSinkWriter.java:121)
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.write(MultiTableSinkWriter.java:150)
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.write(MultiTableSinkWriter.java:43)
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:269)
... 16 more
Caused by: java.lang.NullPointerException
at org.apache.seatunnel.connectors.seatunnel.redis.sink.RedisSinkWriter.handleOtherTypes(RedisSinkWriter.java:223)
at org.apache.seatunnel.connectors.seatunnel.redis.sink.RedisSinkWriter.getValue(RedisSinkWriter.java:178)
at org.apache.seatunnel.connectors.seatunnel.redis.sink.RedisSinkWriter.write(RedisSinkWriter.java:121)
at org.apache.seatunnel.connectors.seatunnel.redis.sink.RedisSinkWriter.write(RedisSinkWriter.java:55)
at org.apache.seatunnel.api.sink.multitablesink.MultiTableWriterRunnable.run(MultiTableWriterRunnable.java:62)
... 5 more

Does this PR introduce any user-facing change?

no

How was this patch tested?

exist tests

Check list

@Hisoka-X
Copy link
Member

Hi @lm-ylj , thanks for the pr. Any test case can reproduce this bug?

@github-actions github-actions bot added the e2e label Dec 2, 2024
@lm-ylj
Copy link
Contributor Author

lm-ylj commented Dec 2, 2024

Hi @lm-ylj , thanks for the pr. Any test case can reproduce this bug?

I added some e2e test cases as follows:

  1. normal key is null
  2. custom key is null
  3. when the value type is hash, the hash key or hash value is null
  4. when the value type is other types, value is null

@Hisoka-X
Copy link
Member

Hisoka-X commented Dec 2, 2024

cc @FuYouJ as well

Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lm-ylj

@hailin0 hailin0 merged commit 6b9074e into apache:dev Dec 2, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants