Loki Grafana Alerts are not working via Prometheus datasource

[cyriltovena]

What happened:
In Loki we don't have yet an alerting storing, so we took advantages of the similarities between Loki and Prometheus API to create Grafana Alert on top of Loki but added as a Prometheus datasource, see below.

We even have a video explaining how to do that https://www.youtube.com/watch?v=GdgX46KwKqo

Now it seems that since 6.6 alert are failing to be evaluated, although graphing still works see below:

It seems that something is now parsing the PromQL query and since Loki has some differences it fails. Although the query is actually the same as PromQL (sum(rate({cluster="dev-us-central1", job="loki-boltdb-shipper/distributor", level!~"debug|info"}[5m])) by (level)).

Response from the test endpoint:

{"firing":true,"state":"pending","conditionEvals":" = true","timeMs":"12.647ms","error":"tsdb.HandleRequest() error bad_response: readObjectStart: expect { or n, but found p, error found in grafana/grafana#1 byte of ...|parse error|..., bigger context ...|parse error : syntax error: unexpected $end|...","logs":[{"message":"Condition[0]: Query","data":{"from":1588256991975,"queries":[{"refId":"A","model":{"expr":"sum(rate({cluster=\"dev-us-central1\", job=\"loki-boltdb-shipper/distributor\", level!~\"debug|info\"}[5m])) by (level)","interval":"","intervalFactor":3,"legendFormat":"{{level}}","refId":"A"},"datasource":{"id":3,"name":"Loki-Prometheus"},"maxDataPoints":0,"intervalMs":0}],"to":1588257291975}}]}

What you expected to happen:

Alert to still works with Loki Prometheus datasource.

How to reproduce it (as minimally and precisely as possible):

See the video.

Environment:

• Grafana version: seems to break from 6.6 and forward
• Data source type & version: Loki as Prometheus datasource

/cc @aocenas

[cyriltovena]

So the root cause of this is the frontend, not handling POST and we Grafana is using that.