-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Ingest Manager] Basic Elastic Agent documentation #19030
Changes from all commits
846b029
4135449
66bd030
9d7aee1
f6a4a7d
f2673e0
0003890
f0a9dc1
de8baa4
250e8e7
ce87631
b2d2657
84c2ea9
d321304
d6197fa
90f0980
780c4aa
c5522a4
8e6ea52
96a3ff5
4cc6312
1950232
1bc2c22
371d8ed
d946757
c7e162b
62bc73f
1fa71f1
5fd8923
87266ad
75b43c7
d210096
4e2221d
335ebe8
67d1877
6e6a4ab
4abc7f8
cd875fd
f17b3f3
bf55bf6
97c416b
c6a499e
7539961
71a5e81
69c6e23
8a437af
37cab7b
683fe89
816e5b0
d53710b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,78 +1,242 @@ | ||
Agent's notes | ||
|
||
|
||
[[requirements]] | ||
Requirements | ||
|
||
The remote Kibana version of the Agent must be equal or higher than the Agent version, if this is not met | ||
Kibana will refuse the connection. | ||
|
||
|
||
[[Supported Actions by the Agent]] | ||
|
||
The Elastic Agent supports the following actions received by Fleet. | ||
|
||
Implemented: | ||
|
||
[[Configuration change]] | ||
|
||
The Elastic Agent receives a configuration encoded as a JSON object. | ||
|
||
[source,json] | ||
------------------------------------------------------------------------------ | ||
{ | ||
"type": "CONFIG_CHANGE", | ||
"id": "id1", | ||
"data": { | ||
"config": { | ||
"id": "policy-id", | ||
"outputs": { | ||
"default": { | ||
"hosts": "https://localhost:9200" | ||
} | ||
}, | ||
"datasources": [{ | ||
"id": "string", | ||
"enabled": true, | ||
"use_output": "default", | ||
"inputs": [{ | ||
"type": "logs", | ||
"streams": [{ | ||
"paths": ["/var/log/hello.log"] | ||
}] | ||
}] | ||
}] | ||
} | ||
} | ||
} | ||
------------------------------------------------------------------------------ | ||
|
||
[[Planned but not implemented]] | ||
|
||
[source,json] | ||
------------------------------------------------------------------------------ | ||
{ | ||
"type": "UPGRADE", | ||
"id": "id1", | ||
"target_version": "7.2.1" | ||
} | ||
------------------------------------------------------------------------------ | ||
|
||
[source,json] | ||
------------------------------------------------------------------------------ | ||
{ | ||
"type": "DOWNGRADE", | ||
"id": "id1", | ||
"target_version": "7.2.1" | ||
} | ||
------------------------------------------------------------------------------ | ||
|
||
[source,json] | ||
------------------------------------------------------------------------------ | ||
{ | ||
"type": "RELAY", | ||
"id": "id1", | ||
"destination": {"name":"endpoint"} | ||
"data": {} | ||
} | ||
------------------------------------------------------------------------------ | ||
[[elastic-agent-installation-configuration]] | ||
== Get started with {beatname_uc} | ||
|
||
++++ | ||
<titleabbrev>Get started</titleabbrev> | ||
++++ | ||
|
||
Elastic Agent is a single, unified agent that you can deploy to hosts or containers to collect data and send it to the {stack}. Behind the scenes, Elastic Agent runs the {beats} shippers or Endpoint required for your configuration. | ||
|
||
* <<elastic-agent-installation>> | ||
* <<elastic-agent-execution-modes>> | ||
* <<elastic-agent-cmd-options>> | ||
* <<elastic-agent-configuration>> | ||
|
||
[[elastic-agent-installation]] | ||
== Install Elastic Agent | ||
|
||
=== Step 1: Unpack archive | ||
|
||
|
||
[[mac]] | ||
*mac:* | ||
dedemorton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
ifeval::["{release-state}"=="unreleased"] | ||
|
||
Version {version} of {beatname_uc} has not yet been released. | ||
|
||
endif::[] | ||
|
||
ifeval::["{release-state}"!="unreleased"] | ||
|
||
["source","sh",subs="attributes,callouts"] | ||
------------------------------------------------ | ||
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-{version}-darwin-x86_64.tar.gz | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's a bit odd to have step 1 and the curl command. I'd choose one or the other, not both. |
||
tar xzvf elastic-agent-{version}-darwin-x86_64.tar.gz | ||
------------------------------------------------ | ||
|
||
endif::[] | ||
|
||
[[linux]] | ||
*linux:* | ||
|
||
ifeval::["{release-state}"=="unreleased"] | ||
|
||
Version {version} of {beatname_uc} has not yet been released. | ||
|
||
endif::[] | ||
|
||
ifeval::["{release-state}"!="unreleased"] | ||
|
||
["source","sh",subs="attributes,callouts"] | ||
------------------------------------------------ | ||
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-{version}-linux-x86_64.tar.gz | ||
tar xzvf elastic-agent-{version}-linux-x86_64.tar.gz | ||
------------------------------------------------ | ||
|
||
endif::[] | ||
|
||
[[win]] | ||
*win:* | ||
|
||
ifeval::["{release-state}"=="unreleased"] | ||
|
||
Version {version} of {beatname_uc} has not yet been released. | ||
|
||
endif::[] | ||
|
||
ifeval::["{release-state}"!="unreleased"] | ||
|
||
. Download the Elastic Agent Windows zip file from the | ||
https://www.elastic.co/downloads/beats/elastic-agent[downloads page]. | ||
|
||
. Extract the contents of the zip file into `C:\Program Files`. | ||
|
||
. Rename the `elastic-agent-<version>-windows` directory to `Elastic-Agent`. | ||
|
||
. Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select *Run As Administrator*). | ||
|
||
. From the PowerShell prompt, run the following commands to install Filebeat as a | ||
Windows service: | ||
+ | ||
[source,shell] | ||
---------------------------------------------------------------------- | ||
PS > cd 'C:\Program Files\Elastic-Agent' | ||
PS C:\Program Files\Elastic-Agent> .\install-service-elastic-agent.ps1 | ||
---------------------------------------------------------------------- | ||
|
||
NOTE: If script execution is disabled on your system, you need to set the execution policy for the current session to allow the script to run. For example: `PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-elastic-agent.ps1`. | ||
|
||
endif::[] | ||
|
||
=== Step 2: Run Elastic Agent | ||
|
||
If Elastic Agent is not installed as an auto-starting service, start it manually: | ||
|
||
|
||
[source,shell] | ||
---------------------------------------------------------------------- | ||
$ ./elastic-agent run | ||
---------------------------------------------------------------------- | ||
|
||
[[elastic-agent-execution-modes]] | ||
== Execution modes | ||
dedemorton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Elastic Agent runs in two modes: standalone or fleet. The two modes differ in how you configure and manage the agent. | ||
[float] | ||
=== Standalone mode | ||
|
||
With _standalone mode_, you manually configure and manage the agent locally. Each agent is configured to be in standalone mode by default after installation. | ||
At startup, Elastic Agent reads the configuration file specified by the `-c` argument or uses the default configuration, `elastic-agent.yml`, which is located in the same directory as the agent. | ||
|
||
For configuration options see `elastic-agent_configuration_example.yml` | ||
|
||
=== Fleet mode | ||
|
||
With _fleet mode_, you manage Elastic Agent remotely. The agent uses a trusted {kib} instance to retrieve configurations and report agent events. This trusted {kib} instance must have Ingest Manager and Fleet enabled. | ||
|
||
To create a trusted communication channel between Elastic Agent and {kib}, you enroll the agent to Fleet. | ||
|
||
To enroll an Elastic Agent to Fleet: | ||
|
||
|
||
michalpristas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
. Stop the agent. | ||
|
||
michalpristas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
. Enroll the agent: | ||
+ | ||
[source,shell] | ||
---------------------------------------------------------------------- | ||
$ ./elastic-agent http://localhost:5601 $token | ||
dedemorton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
---------------------------------------------------------------------- | ||
+ | ||
`$token` is an enrollment token acquired from Fleet. | ||
|
||
[[elastic-agent-cmd-options]] | ||
== Command line options | ||
|
||
The `elastic-agent run` command provides flags that alter the behavior of an agent. | ||
|
||
==== `-path.home` | ||
dedemorton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The home directory of the Elastic Agent. `path.home` determines the location of the configuration files and data directory. | ||
|
||
==== `-c` | ||
|
||
The configuration file to load. | ||
If not specified, Elastic Agent uses `{path.home}/elastic-agent.yml`. | ||
|
||
|
||
==== `-path.data` | ||
|
||
The data directory used by Elastic Agent to store downloaded artifacts. Also stores logs for any Beats started and managed by Elastic Agent. | ||
|
||
If not specified, Elastic Agent uses `{path.home}/data`. | ||
|
||
[[elastic-agent-configuration]] | ||
== Configure Elastic Agent | ||
|
||
By default Elastic Agent runs in standalone mode to ingest system data and send it to a local {es} instance running on port 9200. It uses the demo credentials of the `elastic` user. It's also configured to monitor all Beats managed by the agent and send the Beats logs and metrics to the same {es) instance. | ||
|
||
To alter this behavior, configure the output. | ||
|
||
=== Configure the output | ||
|
||
michalpristas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Elastic Agent enables definition of multiple outputs where each data source can be paired with different output. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we can say this more concisely. How about:
There are still a couple of problems with this. Will users know what a data source is? What do you mean by pairing? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sorry, my comment had some typos. Fixed. :-P |
||
|
||
At the moment Elastic Agent works only with Elasticsearch output. | ||
Sample configuration can look like the example below: | ||
|
||
[source,yaml] | ||
------------------------------------------------------------------------------------- | ||
outputs: | ||
default: | ||
type: elasticsearch | ||
hosts: [127.0.0.1:9200] | ||
username: elastic | ||
password: changeme | ||
monitoring: | ||
type: elasticsearch | ||
api_key: VuaCfGcBCdbkQm-e5aOx:ui2lp2axTNmsyakw9tvNnw | ||
hosts: ["localhost:9200"] | ||
ca_sha256: "7lHLiyp4J8m9kw38SJ7SURJP4bXRZv/BNxyyXkCcE/M=" | ||
------------------------------------------------------------------------------------- | ||
|
||
This example configures two outputs: `default` and `monitoring`. | ||
Notice that they use different authentication methods. The first one uses a username and password pair, and the second one contains an api key. | ||
|
||
[NOTE] | ||
============== | ||
A default output configuration is required. | ||
============== | ||
|
||
=== Configure Beats monitoring | ||
|
||
michalpristas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Elastic Agent is monitoring _Beats_ by default. To disable or change monitoring settings, set options under `settings.monitoring`: | ||
|
||
[source,yaml] | ||
------------------------------------------------------------------------------------- | ||
settings.monitoring: | ||
# enabled turns on monitoring of running processes | ||
enabled: true | ||
# enables log monitoring | ||
logs: true | ||
# enables metrics monitoring | ||
metrics: true | ||
# specifies output to be used | ||
use_output: monitoring | ||
------------------------------------------------------------------------------------- | ||
|
||
|
||
To disable monitoring, set `settings.monitoring.enabled` to `false`. When set to `false`, Beats monitoring is turned off, and all other options in this section are ignored. | ||
To enable monitoring, set `settings.monitoring.enabled` to `true`. Also set the `logs` and `metrics` settings to control whether logs, metrics, or both are collected. If neither setting is specified, monitoring is disabled. | ||
|
||
|
||
Set `use_output` to specify the output to which monitoring events are sent. | ||
|
||
=== Specify data sources | ||
|
||
By default Elastic Agent collects system metrics, such as cpu, memory, network, and filesystem metrics, and sends them to the default output. For example: | ||
|
||
|
||
[source,yaml] | ||
------------------------------------------------------------------------------------- | ||
datasources: | ||
- namespace: default | ||
use_output: default | ||
inputs: | ||
- type: system/metrics | ||
streams: | ||
- metricset: cpu | ||
dataset: system.cpu | ||
- metricset: memory | ||
dataset: system.memory | ||
- metricset: network | ||
dataset: system.network | ||
- metricset: filesystem | ||
dataset: system.filesystem | ||
------------------------------------------------------------------------------------- | ||
|
||
If `use_output` is not specified, the `default` output is used. | ||
|
||
For more examples, see `elastic-agent_configuration_example.yml` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@michalpristas I think the requirement is that the Agent must be equal or higher than the Agent minor version. I think we are still able to make patch releases for Agent without updating the whole stack. Can you confirm?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we removed sending kbn-version as there was a check on kibana side which required it to be
==
when sent. not sure what the status of the check is atm cc @nchaulet