Skip to content

Commit

Permalink
[UII] Add status tracking for agentless integrations (#199567)
Browse files Browse the repository at this point in the history 
## Summary

Resolves elastic/ingest-dev#3933. For
deployments that support agentless, integrations with agentless
deployment mode enabled will allow the status of agentless integration
policies to be tracked.

### Key technical changes

- A new field `supports_agentless` was added to package policies. This
field already exists on agent policies. When an agentless integration is
created, `supports_agentless: true` is now added to both the package
policy and its parent agent policy.
- This allows easier filtering for agentless integrations as we avoid
having to retrieve & check against every parent agent policy.
- This also means existing agentless policies do not get this new status
tracking UI, only new ones created after this change. Since agentless is
not yet GA, I think this is okay.
- `/api/fleet/agent_status/data` now takes optional query params
`pkgName` and `pkgVersion`. When both are specified, the API will check
if agent(s) have ingested data for only that package's datastreams.

## UI walkthrough
<details>
<summary>🖼️ Click to show screenshots</summary>

1. **Integration policies** page now shows two tables for integrations
meeting the above condition, one for agentless policies and one for
agent-based policies:


![image](https://github.com/user-attachments/assets/58c6a932-9bda-4229-ba5f-d341bdbd539a)

2. Clicking the status badge in the agentless policies table opens a
flyout with two steps: confirm agentless enrollment and confirm incoming
data:


![image](https://github.com/user-attachments/assets/e19e6ba0-f40d-48a7-a524-0373934ac46a)

3. Confirm agentless enrollment polls for an agent enrolled into that
integration policy's agent policy. If that agent is reporting an
unhealthy status, the integration component UI is shown. This UI is the
same one used on Fleet > Agents > Agent details page and shows all
components reported by that agent:


![image](https://github.com/user-attachments/assets/ce214f7f-4bdd-48e5-a5eb-a1e8fcc7a512)

4. Once a healthy agentless enrollment is established, confirm incoming
data starts polling for data for that integration ingested by that agent
ID in the past 5 minutes:


![image](https://github.com/user-attachments/assets/7f3de40b-3418-4174-b529-e805407949b6)

5. If data could not be retrieved in 5 minutes, an error message shows
while polling continues in the background:


![image](https://github.com/user-attachments/assets/a3fd198e-1570-4357-9b7f-e541a769d33f)

6. If data is retrieved, a success message is shown:


![image](https://github.com/user-attachments/assets/f4e442af-ca60-4448-9bfb-3f244cd03c2d)
</details>

## Testing
Easiest way to test is use the Cloud deployment from this PR. Enable
Beta integrations and navigate to CSPM. Add a CSPM integration using
`Agentless` setup technology. Then you can track the status of the
agentless deployment on the Integrations policies tab.

For local testing, the following is required to simulate agentless
agent:
1. Add the following to kibana.dev.yml:
```
xpack.cloud.id: 'anything-to-pass-cloud-validation-checks'
xpack.fleet.agentless.enabled: true
xpack.fleet.agentless.api.url: 'https://localhost:8443'
xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt'
xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key'
xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt'
```
2. Apply [this
patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to prevent attempt to create agentless pod
3. Enroll a Fleet Server as usual
4. Enable Beta integrations and navigate to CSPM. Add a CSPM integration
using `Agentless` setup technology.
5. Enroll a normal Elastic Agent to the agent policy for that CSPM
integration by using the token from Enrollment tokens

## To-do
- [x] API tests
- [x] Unit UI tests
- [x] Manual Cloud tests
- [x] File docs request
  - elastic/ingest-docs#1466
- [ ] Update troubleshooting guide link once available

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
@jen-huang @kibanamachine
jen-huang and kibanamachine authored Nov 26, 2024
1 parent f2da55a commit 3188cda
Show file tree
Hide file tree
Showing 46 changed files with 2,659 additions and 590 deletions.
118 changes: 118 additions & 0 deletions oas_docs/bundle.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6919,6 +6919,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -7943,6 +7949,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -8736,6 +8748,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -9790,6 +9808,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -10813,6 +10837,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -11607,6 +11637,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -12744,6 +12780,22 @@
]
}
},
{
"in": "query",
"name": "pkgName",
"required": false,
"schema": {
"type": "string"
}
},
{
"in": "query",
"name": "pkgVersion",
"required": false,
"schema": {
"type": "string"
}
},
{
"in": "query",
"name": "previewData",
Expand Down Expand Up @@ -30076,6 +30128,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -30563,6 +30621,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"vars": {
"additionalProperties": {
"additionalProperties": false,
Expand Down Expand Up @@ -30800,6 +30864,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"vars": {
"additionalProperties": {
"anyOf": [
Expand Down Expand Up @@ -31336,6 +31406,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -32038,6 +32114,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -33210,6 +33292,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -33622,6 +33710,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"vars": {
"additionalProperties": {
"additionalProperties": false,
Expand Down Expand Up @@ -34314,6 +34408,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down Expand Up @@ -34808,6 +34908,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"vars": {
"additionalProperties": {
"additionalProperties": false,
Expand Down Expand Up @@ -35044,6 +35150,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"vars": {
"additionalProperties": {
"anyOf": [
Expand Down Expand Up @@ -35579,6 +35691,12 @@
},
"type": "array"
},
"supports_agentless": {
"default": false,
"description": "Indicates whether the package policy belongs to an agentless agent policy.",
"nullable": true,
"type": "boolean"
},
"updated_at": {
"type": "string"
},
Expand Down
Loading

0 comments on commit 3188cda

Please sign in to comment.