prometheus-rspamd-exporter broken on 22.11 and unstable

[whentze]

Describe the bug

The config.yml for json_exporter produced by NixOS' prometheus-rspamd-exporter module does not work with json_exporter 0.5.0, which is the version shipped by NixOS 22.11 and current unstable.

There was a breaking change in their config format: prometheus-community/json_exporter#166
Now, a top-level modules key is needed, and all config files for json_exporter 0.4.0 and prior are broken.

Steps To Reproduce

Steps to reproduce the behavior:

1. enable services.prometheus.exporters.rspamd and configure it
2. start the service
3. try to get some metrics from the exporter, e.g. via curl localhost:7980/probe

Instead of returning metrics, the exporter answers with a HTTP 400 and Unknown module "default".

Expected behavior

The exporter should return metrics, as it did on NixOS 22.05 with json_exporter 0.4.0.

Screenshots

Log output from starting the exporter on 22.05:

ts=2023-01-12T16:29:09.721Z caller=main.go:52 level=info msg="Starting json_exporter" version="(version=, branch=, revision=)"
ts=2023-01-12T16:29:09.721Z caller=main.go:53 level=info msg="Build context" build="(go=go1.17.13, user=, date=)"
ts=2023-01-12T16:29:09.721Z caller=main.go:55 level=info msg="Loading config file" file=/nix/store/hll3a1r19z43vfykr6dvfkjxn6hjpiaa-rspamd-exporter-config.yml
ts=2023-01-12T16:29:09.724Z caller=main.go:65 level=info msg="Loaded config file" config="{\"Headers\":null,\"Metrics\":[{\"Name\":\"rspamd_actions_add_header\",\"Path\":\"{ .actions['add\\\\ header'] }\" [... lots more config] }}"
ts=2023-01-12T16:29:09.725Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false

And on 22.11:

ts=2023-01-12T16:54:05.128Z caller=main.go:53 level=info msg="Starting json_exporter" version="(version=, branch=, revision=)"
ts=2023-01-12T16:54:05.128Z caller=main.go:54 level=info msg="Build context" build="(go=go1.19.4, user=, date=)"
ts=2023-01-12T16:54:05.128Z caller=main.go:56 level=info msg="Loading config file" file=/tmp/config.json
ts=2023-01-12T16:54:05.129Z caller=main.go:66 level=info msg="Loaded config file" config="{\"Modules\":null}"
ts=2023-01-12T16:54:05.129Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false

Note how 0 modules get loaded.

Additional context

It's possible that other exporters that are based on json_exporter also need fixing.

Notify maintainers

@WilliButz

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.78-hardened1, NixOS, 22.11 (Raccoon), 22.11pre-git`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - nixpkgs: `/nix/store/phahf796d61dbvz1xvj6yfncjmpvq6sn-ha04c37d5m2p0cmb3cgfbp9m9qv2m3gi-nixos-nixpkgs-e9ade2c8240e00a4784fac282a502efff2786bdc`

[whentze]

I think the easiest/quickest way to fix this is probably to just wrap the current config in

{
  "modules": {
    "default": ?
  }
}

That would fix the immediate issue AFAICT.

[Radvendii]

Could probably change this line

nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/rspamd.nix

Line 12 in 21fa97f

metrics = (map (path: {

to

 modules.default.metrics = (map (path: {

This is really hard to patch after-the-fact outside of Nixpkgs without copy-pasting basically the whole module. This is the module I would think to write as a workaround:

long code block

{ lib, config, pkgs, ... }: {
  systemd.services.prometheus-rpsamd-exporter.serviceConfig.ExecStart = let
    cfg = config.services.prometheus.exporters.rspamd;

    mkFile = conf:
      pkgs.writeText "rspamd-exporter-config.yml" (builtins.toJSON conf);

    generateConfig = extraLabels: {
      modules.default.metrics = (map (path: {
        name = "rspamd_${replaceStrings [ "[" "." " " "]" "\\" "'" ] [ "_" "_" "_" "" "" "" ] path}";
        path = "{ .${path} }";
        labels = extraLabels;
      }) [
        "actions['add\\ header']"
        "actions['no\\ action']"
        "actions['rewrite\\ subject']"
        "actions['soft\\ reject']"
        "actions.greylist"
        "actions.reject"
        "bytes_allocated"
        "chunks_allocated"
        "chunks_freed"
        "chunks_oversized"
        "connections"
        "control_connections"
        "ham_count"
        "learned"
        "pools_allocated"
        "pools_freed"
        "read_only"
        "scanned"
        "shared_chunks_allocated"
        "spam_count"
        "total_learns"
      ]) ++ [{
        name = "rspamd_statfiles";
        type = "object";
        path = "{.statfiles[*]}";
        labels = recursiveUpdate {
          symbol = "{.symbol}";
          type = "{.type}";
        } extraLabels;
        values = {
          revision = "{.revision}";
          size = "{.size}";
          total = "{.total}";
          used = "{.used}";
          languages = "{.languages}";
          users = "{.users}";
        };
      }];
    };
  in
   lib.mkForce ''
      ${pkgs.prometheus-json-exporter}/bin/json_exporter \
        --config.file ${mkFile (generateConfig cfg.extraLabels)} \
        --web.listen-address "${cfg.listenAddress}:${toString cfg.port}" \
        ${concatStringsSep " \\\n  " cfg.extraFlags}
  '';
}

[whentze]

Fixed by #208715.

For some reason I hadn't found that PR when I searched on GitHub while making this issue.