-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Amtool implementation #636
Merged
Merged
Changes from all commits
Commits
Show all changes
48 commits
Select commit
Hold shift + click to select a range
3ea3e8a
Implement alertmanager cli tool 'amtool'
cd0d337
Update gitignore
7578ffc
Implement config subcommand
f33bac9
Move output flag to root command. Also print config notification to s…
b2ab06b
Implement alert command
be80a7e
Revamp silence command
02c4ac2
Flesh out format stuff
ba742c8
Fix config format for extended output
e61d360
Allow alername default when no = specified
605c403
Vendor in cobra, viper, and pflag
fbd99d9
Fix argument ordering
0d534c2
added more comments:
89869e7
Remove unneeded copyright block
a0bab45
Fix capital A in Alertmanager
1d8dbf0
Exit 1 instead of -1
513312f
Remove un-used function
8915d2f
Fixup tests to simplified types
a22ac3d
Remove unneeded comment from code generation
d9413d0
Make alertmanager url flag consistent with alertmanager cli syntax
d1dec7e
Made the url validation better
360a631
Fixups from 1:1 session
2199e68
Vastly improve help commands
8f614b0
Add docs about config file
8e8f364
Add docs about how to generate man pages and bash completions
597a362
Merge pull request #1 from Kellel/amtool
Kellel c7e603a
Fix my vendors
f5478c6
Re-order things in tests so that they pass (things are sorted now)
e523382
Fix ordering of regex string formatter
f536f83
Fix fmt.Sprintf formatting to multiple lines so that it's easier to read
eaf6249
Refactor config file loading
fbdc789
Update docs for config file
0aa1b09
Small fixes to config system
7326c32
fixup name of VersionInfo
576e643
Merge changes from master
70316d8
Allow export of regex from label parser
fba0793
Merge in changes related to parser code
32a8973
Major refactor to use filters in #633
aa47c41
Add missing dependancy
e3caaa8
Add alert query alias
7d6a860
Re-implement `alertname=` prefix when the first argument isn't a matcher
5d0bba4
Fix bug with extra regex information leaking from cli created silences
ac50b07
Implement `-q` flag for only silence id output
507b32b
Revert "Fix bug with extra regex information leaking from cli created…
dabbd06
Revert change to vendored repo and apply parsing change patch
7f3ce6d
Make `-q` `--quiet` flags a silence global command
0da7098
Fixup config handling since things changed in alertmanager since initail
ffb6747
Remove double error notifications
e56573b
Merge branch 'master' into master
Kellel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
/data/ | ||
/alertmanager | ||
/amtool | ||
*.yml | ||
*.yaml | ||
/.build | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Generating amtool artifacts | ||
|
||
Amtool comes with the option to create a number of eaze-of-use artifacts that can be created. | ||
|
||
go run generate_amtool_artifacts.go | ||
|
||
## Bash completion | ||
|
||
The bash completion file can be added to `/etc/bash_completion.d/`. | ||
|
||
## Man pages | ||
|
||
Man pages can be added to the man directory of your choice | ||
|
||
cp artifacts/*.1 /usr/local/share/man/man1/ | ||
sudo mandb | ||
|
||
Then you should be able to view the man pages as expected. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
package main | ||
|
||
import ( | ||
"github.com/spf13/cobra/doc" | ||
|
||
"github.com/prometheus/alertmanager/cli" | ||
) | ||
|
||
func main() { | ||
cli.RootCmd.GenBashCompletionFile("amtool_completion.sh") | ||
header := &doc.GenManHeader{ | ||
Title: "amtool", | ||
Section: "1", | ||
} | ||
|
||
doc.GenManTree(cli.RootCmd, header, ".") | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
package cli | ||
|
||
import ( | ||
"encoding/json" | ||
"errors" | ||
"fmt" | ||
"net/http" | ||
"net/url" | ||
"path" | ||
"strings" | ||
"time" | ||
|
||
"github.com/prometheus/alertmanager/cli/format" | ||
"github.com/prometheus/alertmanager/dispatch" | ||
"github.com/prometheus/alertmanager/pkg/parse" | ||
"github.com/prometheus/common/model" | ||
"github.com/spf13/cobra" | ||
"github.com/spf13/viper" | ||
) | ||
|
||
type alertmanagerAlertResponse struct { | ||
Status string `json:"status"` | ||
Data []*alertGroup `json:"data,omitempty"` | ||
ErrorType string `json:"errorType,omitempty"` | ||
Error string `json:"error,omitempty"` | ||
} | ||
|
||
type alertGroup struct { | ||
Labels model.LabelSet `json:"labels"` | ||
GroupKey uint64 `json:"groupKey"` | ||
Blocks []*alertBlock `json:"blocks"` | ||
} | ||
|
||
type alertBlock struct { | ||
RouteOpts interface{} `json:"routeOpts"` | ||
Alerts []*dispatch.APIAlert `json:"alerts"` | ||
} | ||
|
||
// alertCmd represents the alert command | ||
var alertCmd = &cobra.Command{ | ||
Use: "alert", | ||
Short: "View and search through current alerts", | ||
Long: `View and search through current alerts. | ||
|
||
Amtool has a simplified prometheus query syntax, but contains robust support for | ||
bash variable expansions. The non-option section of arguments constructs a list | ||
of "Matcher Groups" that will be used to filter your query. The following | ||
examples will attempt to show this behaviour in action: | ||
|
||
amtool alert query alertname=foo node=bar | ||
|
||
This query will match all alerts with the alertname=foo and node=bar label | ||
value pairs set. | ||
|
||
amtool alert query foo node=bar | ||
|
||
If alertname is ommited and the first argument does not contain a '=' or a | ||
'=~' then it will be assumed to be the value of the alertname pair. | ||
|
||
amtool alert query 'alertname=~foo.*' | ||
|
||
As well as direct equality, regex matching is also supported. The '=~' syntax | ||
(similar to prometheus) is used to represent a regex match. Regex matching | ||
can be used in combination with a direct match. | ||
`, | ||
RunE: queryAlerts, | ||
} | ||
|
||
var alertQueryCmd = &cobra.Command{ | ||
Use: "query", | ||
Short: "View and search through current alerts", | ||
Long: alertCmd.Long, | ||
RunE: queryAlerts, | ||
} | ||
|
||
func init() { | ||
RootCmd.AddCommand(alertCmd) | ||
alertCmd.AddCommand(alertQueryCmd) | ||
alertQueryCmd.Flags().Bool("expired", false, "Show expired alerts as well as active") | ||
alertQueryCmd.Flags().BoolP("silenced", "s", false, "Show silenced alerts") | ||
viper.BindPFlag("expired", alertQueryCmd.Flags().Lookup("expired")) | ||
viper.BindPFlag("silenced", alertQueryCmd.Flags().Lookup("silenced")) | ||
} | ||
|
||
func fetchAlerts(filter string) ([]*dispatch.APIAlert, error) { | ||
alertResponse := alertmanagerAlertResponse{} | ||
|
||
u, err := GetAlertmanagerURL() | ||
if err != nil { | ||
return []*dispatch.APIAlert{}, err | ||
} | ||
|
||
u.Path = path.Join(u.Path, "/api/v1/alerts/groups") | ||
u.RawQuery = "filter=" + url.QueryEscape(filter) | ||
|
||
res, err := http.Get(u.String()) | ||
if err != nil { | ||
return []*dispatch.APIAlert{}, err | ||
} | ||
|
||
defer res.Body.Close() | ||
|
||
err = json.NewDecoder(res.Body).Decode(&alertResponse) | ||
if err != nil { | ||
return []*dispatch.APIAlert{}, errors.New("Unable to decode json response") | ||
} | ||
|
||
if alertResponse.Status != "success" { | ||
return []*dispatch.APIAlert{}, fmt.Errorf("[%s] %s", alertResponse.ErrorType, alertResponse.Error) | ||
} | ||
|
||
return flattenAlertOverview(alertResponse.Data), nil | ||
} | ||
|
||
func flattenAlertOverview(overview []*alertGroup) []*dispatch.APIAlert { | ||
alerts := []*dispatch.APIAlert{} | ||
for _, group := range overview { | ||
for _, block := range group.Blocks { | ||
alerts = append(alerts, block.Alerts...) | ||
} | ||
} | ||
return alerts | ||
} | ||
|
||
func queryAlerts(cmd *cobra.Command, args []string) error { | ||
expired := viper.GetBool("expired") | ||
showSilenced := viper.GetBool("silenced") | ||
|
||
var filterString = "" | ||
if len(args) == 1 { | ||
// If we only have one argument then it's possible that the user wants me to assume alertname=<arg> | ||
// Attempt to use the parser to pare the argument | ||
// If the parser fails then we likely don't have a (=|=~|!=|!~) so lets prepend `alertname=` to the front | ||
_, err := parse.Matcher(args[0]) | ||
if err != nil { | ||
filterString = fmt.Sprintf("{alertname=%s}", args[0]) | ||
} else { | ||
filterString = fmt.Sprintf("{%s}", strings.Join(args, ",")) | ||
} | ||
} else if len(args) > 1 { | ||
filterString = fmt.Sprintf("{%s}", strings.Join(args, ",")) | ||
} | ||
|
||
fetchedAlerts, err := fetchAlerts(filterString) | ||
if err != nil { | ||
return err | ||
} | ||
|
||
displayAlerts := []*dispatch.APIAlert{} | ||
for _, alert := range fetchedAlerts { | ||
// If we are only returning current alerts and this one has already expired skip it | ||
if !expired { | ||
if !alert.EndsAt.IsZero() && alert.EndsAt.Before(time.Now()) { | ||
continue | ||
} | ||
} | ||
|
||
if !showSilenced { | ||
// If any silence mutes this alert don't show it | ||
if alert.Silenced != "" { | ||
continue | ||
} | ||
} | ||
|
||
displayAlerts = append(displayAlerts, alert) | ||
} | ||
|
||
formatter, found := format.Formatters[viper.GetString("output")] | ||
if !found { | ||
return errors.New("Unknown output formatter") | ||
} | ||
return formatter.FormatAlerts(displayAlerts) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
package cli | ||
|
||
import ( | ||
"encoding/json" | ||
"errors" | ||
"fmt" | ||
"net/http" | ||
"path" | ||
"time" | ||
|
||
"github.com/prometheus/alertmanager/cli/format" | ||
"github.com/prometheus/alertmanager/config" | ||
"github.com/spf13/cobra" | ||
"github.com/spf13/viper" | ||
) | ||
|
||
// Config is the response type of alertmanager config endpoint | ||
// Duped in cli/format needs to be moved to common/model | ||
type Config struct { | ||
Config string `json:"config"` | ||
ConfigJSON config.Config `json:configJSON` | ||
MeshStatus map[string]interface{} `json:"meshStatus"` | ||
VersionInfo map[string]string `json:"versionInfo"` | ||
Uptime time.Time `json:"uptime"` | ||
} | ||
|
||
type MeshStatus struct { | ||
Name string `json:"name"` | ||
NickName string `json:"nickName"` | ||
Peers []PeerStatus `json:"peerStatus"` | ||
} | ||
|
||
type PeerStatus struct { | ||
Name string `json:"name"` | ||
NickName string `json:"nickName"` | ||
UID uint64 `uid` | ||
} | ||
|
||
type alertmanagerStatusResponse struct { | ||
Status string `json:"status"` | ||
Data Config `json:"data,omitempty"` | ||
ErrorType string `json:"errorType,omitempty"` | ||
Error string `json:"error,omitempty"` | ||
} | ||
|
||
// alertCmd represents the alert command | ||
var configCmd = &cobra.Command{ | ||
Use: "config", | ||
Short: "View the running config", | ||
Long: `View current config | ||
|
||
The amount of output is controlled by the output selection flag: | ||
- Simple: Print just the running config | ||
- Extended: Print the running config as well as uptime and all version info | ||
- Json: Print entire config object as json`, | ||
RunE: queryConfig, | ||
} | ||
|
||
func init() { | ||
RootCmd.AddCommand(configCmd) | ||
} | ||
|
||
func fetchConfig() (Config, error) { | ||
configResponse := alertmanagerStatusResponse{} | ||
u, err := GetAlertmanagerURL() | ||
if err != nil { | ||
return Config{}, err | ||
} | ||
|
||
u.Path = path.Join(u.Path, "/api/v1/status") | ||
res, err := http.Get(u.String()) | ||
if err != nil { | ||
return Config{}, err | ||
} | ||
|
||
defer res.Body.Close() | ||
decoder := json.NewDecoder(res.Body) | ||
|
||
err = decoder.Decode(&configResponse) | ||
if err != nil { | ||
panic(err) | ||
return Config{}, err | ||
} | ||
|
||
if configResponse.Status != "success" { | ||
return Config{}, fmt.Errorf("[%s] %s", configResponse.ErrorType, configResponse.Error) | ||
} | ||
|
||
return configResponse.Data, nil | ||
} | ||
|
||
func queryConfig(cmd *cobra.Command, args []string) error { | ||
config, err := fetchConfig() | ||
if err != nil { | ||
return err | ||
} | ||
|
||
formatter, found := format.Formatters[viper.GetString("output")] | ||
if !found { | ||
return errors.New("Unknown output formatter") | ||
} | ||
|
||
c := format.Config(config) | ||
|
||
return formatter.FormatConfig(c) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
package format | ||
|
||
import ( | ||
"io" | ||
"time" | ||
|
||
"github.com/prometheus/alertmanager/config" | ||
"github.com/prometheus/alertmanager/dispatch" | ||
"github.com/prometheus/alertmanager/types" | ||
"github.com/spf13/viper" | ||
) | ||
|
||
const DefaultDateFormat = "2006-01-02 15:04:05 MST" | ||
|
||
// Config representation | ||
// Need to get this moved to the prometheus/common/model repo having is duplicated here is smelly | ||
type Config struct { | ||
Config string `json:"config"` | ||
ConfigJSON config.Config `json:configJSON` | ||
MeshStatus map[string]interface{} `json:"meshStatus"` | ||
VersionInfo map[string]string `json:"versionInfo"` | ||
Uptime time.Time `json:"uptime"` | ||
} | ||
|
||
type MeshStatus struct { | ||
Name string `json:"name"` | ||
NickName string `json:"nickName"` | ||
Peers []PeerStatus `json:"peerStatus"` | ||
} | ||
|
||
type PeerStatus struct { | ||
Name string `json:"name"` | ||
NickName string `json:"nickName"` | ||
UID uint64 `uid` | ||
} | ||
|
||
// Formatter needs to be implemented for each new output formatter | ||
type Formatter interface { | ||
SetOutput(io.Writer) | ||
FormatSilences([]types.Silence) error | ||
FormatAlerts([]*dispatch.APIAlert) error | ||
FormatConfig(Config) error | ||
} | ||
|
||
// Formatters is a map of cli argument name to formatter inferface object | ||
var Formatters map[string]Formatter = map[string]Formatter{} | ||
|
||
func FormatDate(input time.Time) string { | ||
dateformat := viper.GetString("date.format") | ||
return input.Format(dateformat) | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This filtering is something I've also implemented in the new alertmanager-ui. Seems like we should add it as a query string param to the url and do the filtering on the initial query, instead of duplicating our work in different consumers of the api?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. I kinda felt like it was a pain doing all this by hand in the cli tool
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The functionality is already available here: https://github.com/prometheus/alertmanager/blob/master/dispatch/dispatch.go#L132-L134
All that needs to be done is parsing a
?silenced=true
to show silenced alerts, and by default don't return them in the response.