Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to customize how provided Kafka serializer handle null #17691

Closed
loicmathieu opened this issue Jun 4, 2021 · 1 comment
Closed

Allow to customize how provided Kafka serializer handle null #17691

loicmathieu opened this issue Jun 4, 2021 · 1 comment
Assignees
Labels
area/kafka kind/enhancement New feature or request

Comments

@loicmathieu
Copy link
Contributor

Description

Currently, provided Kafka ObjectMapperSerializer (and certainly the JSON-B one, this needs to be checked), serializes a null object to the "null" String. This is perfectly valid and will work fine when deserializing via a JSON based deserializer.

However, the tombstone mechanism usually implemented in Kafka Connect to signal a deletion is based on null payloads.

Someone can also use a JSON based serializer and a String based deserializer (for components not needed the payload information to avoid the cost of serialization), this is a valid use case and in this case a payload with the "null" String will ends up with a non-null String for the null JSON.

So we need a way to control how the provided JSON serializers serialize nulls.

Implementation ideas

Have a way to override the default null handling mechanism to serialize null payload as a null byte[] instead of the "null" String. This is easy to implement on itself but we must enable this mechanism only if some Kafka serializer property is set.
The default null handling is compatible with a full JSON serializer / deserializer chain so it must be kept as is by default.

Introduce a io.quarkus.kafka.json.serializer.null-as-null property that, when enabled, serializes null as a null byte[] instead of the "null" String for null payload.
This will be a Kafka serializer config property not a Quarkus one.

@loicmathieu loicmathieu added the kind/enhancement New feature or request label Jun 4, 2021
@quarkus-bot
Copy link

quarkus-bot bot commented Jun 4, 2021

/cc @cescoffier

@loicmathieu loicmathieu self-assigned this Jun 4, 2021
loicmathieu added a commit to loicmathieu/quarkus that referenced this issue Jul 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kafka kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant