Implement a way to run external scripts

[exekias]

We want to run external processes in some parts of Beats. A few cases we want to support are:

• Running a java jar to collect JMX metrics as a Metricbeat module
• Running Nagios scripts to collect metrics or health status

This issue is minded to implement a common mechanism that we can reuse in several places, with a simple API.

• Communication with the process (when needed) will be done through stdin/stdout
• We foresee using several models of running processes, so we should be able to: start/stop/kill, also be notified if they stop (including exit code)

Security model:

Running external stuff can represent an attack vector so we should put some measures in place to avoid issues, for anything that we run we should require:

• Script/binary must be owned by the same user that is running beats
• Permissions must also be strict: only be writable by the owner (we do the same check for config files)
• We should only run scripts that are located in paths that the user has configured
• We won't allow for this paths config to happen remotely (ie with fleet)
• We should explore dropping privileges when they are not needed, for instance, the code requesting to run the script must communicate the needed capabilities, the rest will be disabled
• In the same sense, it should be allowed to set a different user if beats is running as root

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[andrewvc]

Regarding the last point about dropping privileges. We definitely need to do this for heartbeat, which runs as root by default. Perhaps we could just switch users? Another thought is we could do a check to see if we're running as root by default.

A final thought is that heartbeat no longer really needs to run as root now that we support rootless ping. Maybe it's time to change that.

[jsoriano]

Big +1 to this feature, I think this will be great, specially for heartbeat.

Script/binary must be owned by the same user that is running beats

The idea is to allow to run commands installed explicitly for beats? If not, I think it'd be good if binaries installed by root are also allowed, so Beats can use any command installed on the machine, without the need of being executed as root for that (though this opens the pandora's box of running shells and interpreters). Mentioned privileges dropping could still be used to limit some things in any case.

[exekias]

Perhaps we could just switch users? Another thought is we could do a check to see if we're running as root by default.

Yes, actually I think this was in our initial discussions, I've added that option to the description

The idea is to allow to run commands installed explicitly for beats?

We will probably want both commands installed by beats (ie a JMX converter), and installed by the user. For the later, seems that a good security measure is that the folder where they are located must be included in the config before (ie, having a scripts folder)

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ruflin]

Closing this issue in favor of elastic/elastic-agent#1237 as this makes more sense in the context of Elastic Agent then beats itself. All the technical discussion here still makes sense and should be taken into account for elastic/elastic-agent#1237