Plugin architecture

[graup]

This is a proposal to implement a plugin architecture for aergosvr.

Why plugins?

• non-essential features that not every user needs and private logic
• don't bloat the core codebase → easier to maintain, smaller binary
• more direct access than external code has
• can reuse types and utils

What features does a plugin need?

Apart from registering plugins, we will need some additional features in the core.

• register and init
• high level:
  • subscribe to events, e.g. new block (also, contract events after they get implemented)
  • request data, e.g. hub.RequestFuture(message.ChainSvc, &message.GetBlockByNo{BlockNo: i})
• low level:
  • actor middleware, e.g. for tracing

Plugins could just be actors, that way they could even change some existing behaviors instead of just adding features.

Example use cases

• Database indexer: index transactions into external db
• Monitoring, logging, reacting to events
• Debugging, tracing

Implementation ideas / pseudocode

Go supports plugins. They can be built separately and injected at runtime.

There a working example for a feature that should be a plugin here: #13

config.toml

enableplugins = EsIndexerPlugin, FooBarPlugin

[EsIndexerPlugin]
pluginpath = ./path/to/aergoPluginEsIndexer.so
databaseurl = 127.0.0.1:9200

EsIndexerPlugin/plugin.go
package main

func init() {
  component = NewEsIndexer()
  hub := GetDefaultComponentHub()
  hub.Register(component)
  hub.Subscribe(event.NewBlock, component)
}


EsIndexerPlugin/EsIndexer.go
import (
	"github.com/aergoio/aergo/config"
	"github.com/aergoio/aergo/message"
	"github.com/aergoio/aergo/pkg/component"
	"github.com/aergoio/aergo/types"
)
type EsIndexer struct {
  conf *config.Config
  *component.BaseComponent
  hub *component.ComponentHub
}

EsIndexer.Receive
  switch msg := context.Message().(type) {
    case actor.Started:
      db = openDatabaseConnection(conf.EsIndexerPlugin.databaseurl)
      ...
      blockHeight = GetChainAccessor().GetBestBlock().GetHeader().GetBlockNo()
      ...

    case event.NewBlock:
      // sync new block
      db.index('block', msg.hash, msg.metadata)
      for tx in msg.txsList:
        db.index('tx', tx.hash, tx.metadata)
  }

[graup]

I started playing around with this idea, but it is actually not as easy as I thought. You cannot just import "github.com/aergoio/aergo/types" in a plugin, as that contains a bunch of dependencies, some of which (e.g. grpc -> net/trace) complain when multiple instance of them are loaded in a binary. The actor/component has a similar problem.

In order to make internal types available for other Go packages, the basic types should be usable in a more standalone way.

[graup]

To complicate things further, sharing types between go packages that don't use the exact same sources doesn't work at all: golang/go#18827

This renders go plugins pretty much useless for my use case. We need to think of a different solution.

[graup]

Maybe this approach can make sense: https://github.com/hashicorp/go-plugin
Basically run the plugin in a subprocess and communicate over grpc.

Especially since we're already using grpc.

[graup]

We decided to not pursue this for now. For most cases discussed so far, a standalone program is almost as effective as a plugin. We may revisit this issue later.