Proposal: Embed Kafka change logger in storage

[burdiyan]

Right now, updates to group table's storage are published to Kafka from goka.Context#setValueForKey.

It can be useful to embed this functionality directly into storage. This way one could create custom storage implementations that are also backed by Kafka and use it for other purposes, not only for working with group table.

It may be useful when you want to have some custom workload around storage, but don't want to care about it's safety on disk, and have it backed by Kafka.

Kafka Streams also does it this way. They basically have a wrapper around storage that adds changeling functionality to any storage. And then they allow to access storage from their Processor API in any point of the processing pipeline: https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/ChangeLoggingKeyValueBytesStore.java

[burdiyan]

BTW, Although one could embed emitter into storage and pass it directly to the processor callback, I still think having a generic wrapper for it out of the box can be a good improvement, and also would “free” the context from maintaining the change log.

[frairon]

seems not to be important, as there was no activity for a long time. I'll close it, but feel free to reopen/recreate if you still see the requirement.