3.3.2 23b5fc2 loki-local-config.yaml#L44 causing startup failure

[hpst3r]

Describe the bug

Default install of Loki 3.3.2 23b5fc2 (with default config file) fails to start with config file error:

Feb 07 04:03:27 test-mon-0 loki[27588]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Feb 07 04:03:27 test-mon-0 loki[27588]:   line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Feb 07 04:03:27 test-mon-0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE

After removing line 44 in /etc/loki/config.yml, Loki 3.3.2 no longer fails to start.

Likely related to #15039 "3.3.0 loki-local-config.yaml#L41 causing startup failure"

To Reproduce

Steps to reproduce the behavior:

1. Download or install Loki and the default config file
2. Attempt to launch Loki with flag -config.file ./loki-local-config.yaml (or the default /etc/loki/config.yml)

Replicated with:

• loki-linux-amd64 binary and loki-local-config.yaml from Github releases page and 3.3.2 blob, respectively - loki, version 3.3.2 (branch: release-3.3.x, revision: 23b5fc2c)
• RPM, incl. default config file, installed from Grafana Enterprise RPM repository on two installations of AlmaLinux 9.5 - loki, version 3.3.2 (branch: release-3.3.x, revision: 23b5fc2c)
• Deb, incl. default config file, installed from releases page on Debian Sid/Trixie (WSL2) - loki, version 3.3.2 (branch: release-3.3.x, revision: 23b5fc2c)

Expected behavior

Loki starts and runs with default config

Environment:

• Infrastructure: AlmaLinux 9.5 (Grafana repository, Grafana, Promtail, Loki) on Hyper-V and Debian Sid/trixie on WSL2
• Deployment tool: none, RPM repository, release 3.3.2 deb

Screenshots, Promtail config, or terminal output

https://github.com/grafana/loki/blob/v3.3.2/cmd/loki/loki-local-config.yaml#L44

Quick screengrabs while reviewing:

AlmaLinux 9.5 Hyper-V VM with Loki binary from 3.3.2 release and config file from 3.3.2 blob

[liam@test-mon-1 ~]$ ./loki-linux-amd64 -config.file ./loki-local-config.yaml
failed parsing config: ./loki-local-config.yaml: yaml: unmarshal errors:
  line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file

Debian 12 WSL2 with Loki Deb package

liam@liam-p1g4i-0:~$ /usr/bin/loki -config.file /etc/loki/config.yml
failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
  line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file

AlmaLinux 9.5 Hyper-V VM with Loki RPM from Grafana repo

[liam@test-mon-0 ~]$ systemctl status loki
● loki.service - Loki service
     Loaded: loaded (/etc/systemd/system/loki.service; enabled; preset: disabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2025-02-07 04:22:13 UTC; 1s ago
    Process: 29885 ExecStart=/usr/bin/loki -config.file /etc/loki/config.yml (code=exited, status=1/FAILURE)
   Main PID: 29885 (code=exited, status=1/FAILURE)
        CPU: 74ms

The remaining output is from my first AlmaLinux box.

Setting line 44 (metric_aggregation enabled) to enabled: false is not sufficient, as expected. This results in the same error message:

Feb 07 04:21:19 test-mon-0 loki[29706]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Feb 07 04:21:19 test-mon-0 loki[29706]:   line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Feb 07 04:21:19 test-mon-0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE

Running Loki directly as my user produces the same error message:

[liam@test-mon-0 ~]$ /usr/bin/loki -config.file /etc/loki/config.yml
failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
  line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file

When attempting to run Loki with the flag suggested by the second error message (-config.expand-env=true), Loki still fails to launch with an error about field enabled not found in type aggregation.Config.

[liam@test-mon-0 ~]$ /usr/bin/loki -config.file /etc/loki/config.yml -config.expand-env=true
failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
  line 44: field enabled not found in type aggregation.Config

[hpst3r]

Fix appears to have already been implemented and appears to be slated for 3.4.0. Looked through mentions in #15039 too late. Oops.

c96b750