feat: ADR-038: State change indexing with Kafka #2
Conversation
There was a problem hiding this comment.
Overall this looks good ... one question I have is should the kafka plugin be a default case for these examples or not. A smaller and simplified logger output that goes to trace might be the best for a default case with the file and kafka configurations provided as "commented out alternates" in order to streamline size/complexity...
| @@ -276,6 +276,10 @@ func (fss *FileStreamingService) Close() error { | |||
| return nil | |||
| } | |||
|
|
|||
| func (fss *FileStreamingService) ListenSuccess() <-chan bool { | |||
| panic("implement me") | |||
There was a problem hiding this comment.
Just making the compiler happy.
That makes sense. I'll work on a plugin that sends it to trace. |
|
@iramiller Added a |
|
That trace listener looks like a great default case for users. The simplest possible test case. |
Description
closes: #XXXX
Description
closes: #XXXX
This PR introduces the plugin architecture and an implementation of the StreamingService interface that writes out to Kafka. This work is part of ADR-038
The current implementation writes data out to a single topic with only one partition. This is so we can maintain order of messages. Each message is accompanied by a key that indicates the block number and the type of message.
block-{n}-begin-reqblock-{n}-begin-state-change-{i}block-{n}-begin-resblock-{n}-tx-{i}-reqblock-{n}-tx-{i}-state-change-{i}block-{n}-tx-{i}-resblock-{n}-end-reqblock-{n}-end-state-change-{i}block-{n}-end-resHow data is saved to a topic:
Current Limitations:
ADR-038 - State listening occurs in real time. This push (fire-and-forget) may lead to a degradation in performance when listeners are unable to keep up with events and start to fall behind. In addition, listeners will missed state change events in the following two scenarios:
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
docs/) or specification (x/<module>/spec/)godoccomments.Unreleasedsection inCHANGELOG.mdFiles changedin the Github PR explorerCodecov Reportin the comment section below once CI passes