Two-way integration with Prometheus which will silence alerts in Alertmanager when alerts are ack'ed in the Alerta console and delete silences if those alerts are manually re-opened.
For help, join
Clone the GitHub repo and run:
$ python setup.py install
Or, to install remotely from GitHub run:
$ pip install git+https://github.com/alerta/alerta-contrib.git#subdirectory=plugins/prometheus
Note: If Alerta is installed in a python virtual environment then plugins need to be installed into the same environment for Alerta to dynamically discover them.
Add prometheus to the list of enabled PLUGINS in alertad.conf
server configuration file and set plugin-specific variables either in
the server configuration file or as environment variables.
PLUGINS = ['prometheus']The below settings are configured with reasonable defaults:
ALERTMANAGER_API_URL = 'http://localhost:9093'
ALERTMANAGER_SILENCE_DAYS = 1Robust Perception Demo Example
PLUGINS = ['reject','prometheus']
ALERTMANAGER_API_URL = 'http://demo.robustperception.io:9093' # default=http://localhost:9093
ALERTMANAGER_SILENCE_DAYS = 2 # default=1Restart Alerta API and confirm that the plugin has been loaded and enabled.
Set DEBUG=True in the alertad.conf configuration file and look for log
entries similar to below:
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' found. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:50]
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' not enabled in 'PLUGINS'. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:59]
2016-11-21 10:42:55,572 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Add silence for alertname=DiskFull instance=web01 [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:35]
2016-11-21 10:42:55,582 - requests.packages.urllib3.connectionpool[8268]: INFO - Starting new HTTP connection (1): demo.robustperception.io [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:214]
2016-11-21 10:42:55,677 - requests.packages.urllib3.connectionpool[8268]: DEBUG - "POST /api/v1/silences HTTP/1.1" 200 44 [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:401]
2016-11-21 10:42:55,711 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: 200 - {"status":"success","data":{"silenceId":29}} [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:59]
2016-11-21 10:42:55,715 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Added silenceId 29 to attributes [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:67]
- Alertmanager Silences: https://prometheus.io/docs/alerting/alertmanager/#silences
- Robust Perception On-line Demo: http://demo.robustperception.io:9090/consoles/index.html
Copyright (c) 2016 Nick Satterly. Available under the MIT License.