Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support durable log forwarding in seqcli #318

Open
KodrAus opened this issue Feb 22, 2024 · 1 comment
Open

Support durable log forwarding in seqcli #318

KodrAus opened this issue Feb 22, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@KodrAus
Copy link
Member

KodrAus commented Feb 22, 2024

It would be beneficial to merge Seq Forwarder into seqcli. This would let us share a lot of the log shipping functionality they have in common while enhancing existing seqcli commands related to ingestion with bookmarking from Seq Forwarder. It will simplify how we deploy Seq Forwarder, making its functionality available to seqcli users on Windows or Docker on any platform already supported by seqcli.

The CLI may look something like:

seqcli forwarder run --listen "http://localhost:5341" --storage "/seqcli-logs"

And on Windows, some additional commands for managing installation as a Windows service:

seqcli forwarder install
seqcli forwarder start
seqcli forwarder stop
seqcli forwarder remove

As part of this, we'd like to replace the LMDB-based disk buffer with something bespoke that's easier to manage and doesn't rely on the host to install libraries to support it. The transition from LMDB to the new format will be handled in Seq Forwarder, so existing users can let their old buffers age out before migrating to seqcli.

We also need to overhaul how secrets are managed by seqcli to better support secure deployments on non-Windows platforms.

This work effectively deprecates the Seq Forwarder repository, which will enter maintenance mode.

@KodrAus
Copy link
Member Author

KodrAus commented Feb 29, 2024

Apologies to anybody currently watching this repository and getting swamped by our barely coherent broken PRs to a feature/forwarder branch 😅

We're currently working towards getting an MVP of this into the tree. Once it's in reasonable shape we'll write up some more details about what's there and how it all works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant