[peerswap-plugin] Crash logging

[sputn1ck]

On Crash / Panic peerswap does not print to the cln logs.

[wtogami]

Warren: There must be a way for CLN plugins to capture stderr messages to get into CLN's log?

Decker: Yes, in peerswap: close the stderr FD, create a fd pipe to capture stderr internally and wrap it in JSON log notifications before re-emitting to stdout, so lightningd can capture and process it as normal log events

Warren: "wrap it in JSON log notifications" This part means you need another process that itself launches the wrapped peerswap process?

Decker: No, capturing one's own output is relatively routine, and the python plugin infra does it by default for example, maybe that can serve as inspiration

Decker: Got me curious how I implemented this in pyln, and I'm afraid it's using the monkeypatching support in python to swap out the symbolic name instead of juggling FDs: https://github.com/ElementsProject/lightning/blob/be2523ec63ff20aee4b13bb7b7b00777a88f6749/contrib/pyln-client/pyln/client/plugin.py#L1001-L1004

Related
https://stackoverflow.com/questions/10473800/in-go-how-do-i-capture-stdout-of-a-function-into-a-string#10476304

Warren: Why does CLN not simply capture stderr itself?

Decker: Because that's our emergency comm system, this way plugins and subdaemons can still communicate with the outside world even though lightningd might be blocked

[ZmnSCPxj]

Does Go have cheap threads/greenthreads? If so, you could open a pipe then dup2 the input end to STDERR_FILENO (which closes the previous stderr magically) and then launch a greenthread that reads the output end of the pipe. I can do that in C and I assume Go can do that too.

A complication is that outputting to stdout has to be from a single thread, or else you run the risk of an output from one thread showing up right in the middle of an output from your main thread. If your plugin uses an event system mainloop, you can instead trigger on the output end of the pipe, wrap the text into JSON-RPC log notifications like {"json-rpc": "2.0", "method": "log", "params": {"level": "error", "message": "<your message here>"}}, then schedule the entire JSON text to be output atomically on the stdout. But if you crash then you might not even reach the mainloop event system, so --- shrug.

A final alternative, you could use two processes instead to isolate crashes of the "main" program from the log translator. The log translator is the parent process, the "main" program is the child process. Set up two pipes, fork and in the child dup2 to STDOUT_FILENO and STDERR_FILENO the input ends of the pipes, then in the child start your main program. In the parent process, enter a simple event loop around select and wait for inputs from the output ends of the pipes, making sure to emit entire JSON datums from the stdout output end (you need a simple streaming JSON parser, unfortunately, but all you need is to know when one datum ends, buffering the entire output until the datum ends and you can emit the entire datum on the parent process stdout, to ensure that JSON datums do not get cut). For the stderr output end you buffer until you get a newline and then emit a log on your stdout for it. This does complicate attaching gdb to the correct process though....

[nepet]

I am almost certain that this is done by a8119ad