Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ID for alerts and add alert ID's to silences #681

Closed
HyperDevil opened this issue Apr 2, 2017 · 9 comments
Closed

Add ID for alerts and add alert ID's to silences #681

HyperDevil opened this issue Apr 2, 2017 · 9 comments

Comments

@HyperDevil
Copy link

HyperDevil commented Apr 2, 2017

Hi,

I am trying to parse the API output, for alerts and silences. (in PHP)
The silences seem to have an ID, but does not have the explicit alarm ID attached in the JSON output (if alarm has an ID?).
Also the alarms do not have an unique ID so that i cannot match which alarms are being silences by a certain rule.
Is there an internal connection between the alarms and the silences?

@stuartnelson3
Copy link
Contributor

In the current data model, alerts are ephemeral and do not have an ID assigned to them.

Internally, the first silence that matches (and silences) an alert has its ID added to the alert's data structure in the silenced field.

You can do two things:

  1. You can iterate through the alerts and check for the silenced field. This only shows the first silence that affects the alert, though, and won't show any additional silences that also affect it.
  2. You can take the label matchers defining a silence, and make a request to the alert groups endpoint:
http://alertmanager.example.biz/api/v1/alerts/groups?filter=%7Bfoo%3D%22bar%22%2C%20baz%3D~%22quux%22%7D

(filter == {foo="bar", baz=~"quux"})

And that will return the alerts that match the matchers in filter.

@stuartnelson3
Copy link
Contributor

stuartnelson3 commented Apr 3, 2017

That having been said, we are adding alert ID's as part of the UI/API update that is currently underway. Assigning to @mxinden since he's so keen on it.

Update: We are not adding alert IDs.

@fabxc
Copy link
Contributor

fabxc commented Apr 3, 2017

I'm curious about those alert IDs. What does the current plan look like?

@mxinden
Copy link
Member

mxinden commented Apr 3, 2017

@stuartnelson3: I talked to @fabxc in regards to adding Alert ids with the API update. There are no actions (create, update, delete) needed in the UI where we have to address a specific alert. Thereby adding a specific Alert id is not on the agenda. Doing so would open up many questions in regards to the realisation of eventual consistency inside a mesh network of Alertmanagers.

@HyperDevil Let us know if @stuartnelson3 answer solves your problem.

@HyperDevil
Copy link
Author

@mxinden and @stuartnelson3
Thank you for the answers so far.
I am currently using version 0.4.2 of Alertmanager, and i have silences active. But i cannot seem to find any alerts with silences attached when using the /api/v1/alerts, am i using an old version that does not have this?

The point for me is that i need to make a list of alerts outside of Alertmanager using the API, and i want to filter out the alarms that i silcenced (i dont care about which silence rule, as long as it is silenced)

@stuartnelson3
Copy link
Contributor

0.4.2 has the concept of silenced as part of the alert's data structure. Query the api for all alerts, then filter out the ones that have the silenced field.

@HyperDevil
Copy link
Author

@stuartnelson3 i am quering /api/v1/alerts using my web browser. Then i try to search for the string "silen" and i find no results, while i have several alerts which have been silenced. Strange.

@mxinden
Copy link
Member

mxinden commented Aug 13, 2017

Related to #786

hh pushed a commit to ii/alertmanager that referenced this issue Oct 5, 2017
The github.com/beevik/ntp package was recently updated with some
API changes that broke node_exporter. This commit fetches the
latest version of the ntp package and brings node_exporter in
line with the latest API.
@simonpasquier
Copy link
Member

This issue can be closed. cc @stuartnelson3

Here is the output of the /api/v1/alerts endpoint:

{
  "status": "success",
  "data": [
    {
      "labels": {
        "alertname": "TargetDown",
        "instance": "http://localhost:8081",
        "job": "blackbox",
        "severity": "critical"
      },
      "annotations": {
        "summary": "Target down"
      },
      "startsAt": "2018-03-16T12:06:10.79789965+01:00",
      "endsAt": "2018-03-16T13:43:25.800193318+01:00",
      "generatorURL": "http://simon-laptop:9090/graph?g0.expr=probe_success+%21%3D+1&g0.tab=1",
      "status": {
        "state": "suppressed",
        "silencedBy": [
          "e69e786f-e2ac-462d-a3c2-0855c8ba1bf8"
        ],
        "inhibitedBy": []
      },
      "receivers": [
        "webhook"
      ],
      "fingerprint": "5432f81757da9979"
    }
  ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants