Support plugins in Promtail's pipeline

[rfratto]

It would be great if Promtail supported third-party plugins as a stage in a pipeline. We could do this using Hashicorp's go-plugin library, which would be ideal for cross-language support.

[sh0rez]

While I personally like the idea of go-plugin more, we still might want to consider embedded scripting languages:

• https://github.com/containous/yaegi (basically go, just in interpreted)
• https://github.com/robertkrimen/otto (JS)
• https://github.com/d5/tengo

[pracucci]

From my perspective, plugins and scripting would cover different use cases. Plugins may be used to add pipeline stages without any technical limitation, while scripting would be limited to the scripting language capabilities and its functions library. In this scenario, an embedded scripting language may be eventually implemented as a plugin itself.

For this reason, I would suggest to keep this discussion focused on a plugins system.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[roidelapluie]

What about we promote fluentbit instead? It works well for us and has plugins.

[rfratto]

I definitely don't want to discourage fluentbit, but I personally see fluentbit as ideal only if users are already using it. If a user is exploring Loki and wants plugins, it might be discouraging to see that they have to set up a second piece of software in their infrastructure to enable that.

Just my two cents, other core maintainers might view it differently.
/cc @slim-bean @cyriltovena

[cstyan]

@rfratto I think we can close this?

[rfratto]

Yeah sounds like it's unlikely to happen in Promtail, but Grafana Agent had been discussing a plugin model instead. Closing.