Inconsistent timestamp formats when updating silence leads to silence being expired and recreated rather than updated

[vilkaspilkas]

Problem
When using UI or API to update an existing silence, the condition at https://github.com/prometheus/alertmanager/blob/main/silence/silence.go#L589 is never true because the time formats are different. Here is the log line added to debug:

    if ok {
        level.Info(s.logger).Log("msg", fmt.Sprintf("%+v %+v", prev.StartsAt, sil.StartsAt))
        if canUpdate(prev, sil, now) {
            return sil.Id, s.setSilence(sil, now)

And the output it produced:

2021-12-31 05:12:49.731477353 +0000 UTC 2021-12-31 05:12:49.731 +0000 UTC

As you can see an existing silence (created using the UI) has a timestemp with nanosecond precision (05:12:49.731477353), while the timestamps returned by the api and sent by the UI are with millisecond precision (05:12:49.731).

Changing the condition to just use second precision works every time:

diff --git a/silence/silence.go b/silence/silence.go
index 6f3b177d..ed8f5977 100644
--- a/silence/silence.go
+++ b/silence/silence.go
@@ -586,7 +586,7 @@ func canUpdate(a, b *pb.Silence, now time.Time) bool {
        // Allowed timestamp modifications depend on the current time.
        switch st := getState(a, now); st {
        case types.SilenceStateActive:
-               if !b.StartsAt.Equal(a.StartsAt) {
+               if (b.StartsAt.Unix() - a.StartsAt.Unix()) != 0 {
                        return false
                }
                if b.EndsAt.Before(now) {

What did you do?
Used the UI or API to update the end time of the silence - not changing any other field.

What did you expect to see?
The existing silence to be updated.

What did you see instead? Under which circumstances?
The existing silence was expired and the new silence was created.

[simonpasquier]

Closed by #2816