agave-geyser-plugin-interface datasource

[sonicfromnewyoke]

are you planning to implement such kind of datasource?

it should be pretty similar to how it's already done for yellowstone-grpc datasource

In case you're not familiar with geyser, I can make a PR when I'll have some bandwidth (hopefully in the end of January)

[KellianDev]

We're open to it, we are familiar with Geyser.

A geyser plugin datasource in general sounds a bit vague—cause geyser plugins can be implemented in different manners, in the case of Yellowstone gRPC, it's pretty specific what we connect to, where the data is sourced.

Can you describe a bit more how you view it?

[sonicfromnewyoke]

the main point is to bypass the huge grpc service in between if we speak in the context of Yellowstone.

In my head, the final implementation looks like the next one:

• in datasources you will have a new module agave-geyser-datasource which will have a tokio::sync::mpsc::UnboundedSender to listen to updates.
• decoders should not be modified
• processor every engineer will choose for his own needs. Someone would like to store the parsed data in DB, someone would like to trade right after receiving a signal.

in the examples I'm going to provide the bare-minimal geyser plugin (instead of a regular program with tokio::main entrypoint).

Notes:

• on_load - will spin-up the pipeline
• update_account - pass the account update to the pipeline
• notify_transaction - pass the tx to the pipeline.

Pros:

• faster & cheaper processing of the data
• fewer elements to set-up on the server

Cons:

• a little bit harder to implement in comparison to listening to WS
• requires some understanding of validator anatomy

[sonicfromnewyoke]

@KellianDev do you have any thoughts, suggestions, or notices at this point?

[KellianDev]

Sorry for the delayed answer.

Got you, so here Carbon is ran directly from the implementation struct of the GeyserPlugin, correct?

This makes sense to me. I think there's not that much heavy-lifting for building the actual datasource module, just need a good example as you mentioned.

I like the idea, when would you have time to draft a PR?

[sonicfromnewyoke]

yeah, you got me right. I believe, I can start in a couple of days and will send a draft MR to get feedback from you ASAP to prevent making any "incorrect" moves that are not aligned with a project's roadmap or vision

[KellianDev]

Sounds perfect.

[paahaad]

@sonicfromnewyoke but have you consideration deployment. Since the plugin has to be deployed with the validator, the user would need access to the validator's configuration. That might be a con because not everyone runs their own validator.

Geyser plugins are loaded by the validator at runtime. So the plugin would run within the validator's process.

[KellianDev]

@sonicfromnewyoke but have you consideration deployment. Since the plugin has to be deployed with the validator, the user would need access to the validator's configuration. That might be a con because not everyone runs their own validator.

Geyser plugins are loaded by the validator at runtime. So the plugin would run within the validator's process.

Developers that don't run one can use other data sources—which there are multiple of and which we will keep adding to. This is specifically designed for people that do.