Metrics are not being served

[mcgrizzz]

Describe the bug
With the statistics enabled in the config, the /metrics endpoint serves no response.

To Reproduce
Steps to reproduce the behavior:

1. Enable statistics in the config according to the readme
2. Start fan2go
3. Observe fan2go prints "Starting statistics server..."
4. Try to reach configured end point using curl, web browser, or through Prometheus (http://localhost:9000/metrics)
5. Receive empty response from end point

Expected behavior
The /metrics endpoint should serve a non-empty response that can be fed into Prometheus

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Distro: Arch Linux
• uname -a: Linux eregion 5.19.11-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 24 Sep 2022 18:24:15 +0000 x86_64 GNU/Linux
• sensors -v: sensors version 3.6.0+git with libsensors version 3.6.0+git
• fan2go version: dev

Additional context
Add any other context about the problem here.

[markusressel]

Thx for reporting, however I cannot reproduce the issue.

Running the latest dev version I can see metrics at:
http://localhost:9000/metrics
as well as
http://localhost:9000

This is the relevant configuration:

statistics:
  enabled: true
  port: 9000

[mcgrizzz]

Thank you for your response.

I'm not sure if this helps narrow the focus on the issue, but if I leave my browser loading the metrics url, then start fan2go while it's still loading, the page will load instantly.

It seems it will only load as fan2go is starting. Then, any subsequent requests hang and receive an empty response.

I've attached my config file and a script it references incase there is something relevant there:

fan2go.yaml

# A list of fans to control
fans:
  # A user defined ID.
  # Used for logging only
  - id: CPU_FRONT
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 1
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
    
  - id: FRONT_BOTTOM
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 2
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
  - id: CPU_MIDDLE
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 3
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
  - id: BOTTOM_MIDDLE
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 4
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
  - id: BOTTOM_LEFT
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 5
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
  - id: FRONT_TOP
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 6
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve
  - id: BOTTOM_RIGHT
    # The type of fan configuration, one of: hwmon | file
    hwmon:
      # The platform of the controller which is
      # connected to this fan (see sensor.platform below)
      platform: nct6683-isa-0
      # The index of this fan as displayed by `fan2go detect`
      index: 7
    # Indicates whether this fan should never stop rotating, regardless of
    # how low the curve value is
    neverStop: true
    # The curve ID that should be used to determine the
    # speed of this fan
    curve: gpu_curve

sensors:
  - id: cmd_max_gpu
    cmd:
      # Path to the executable to run to retrieve the current sensor value
      exec: /usr/bin/bash
      # (optional) arguments to pass to the executable
      args: [ '/home/celebrimbor/max-gpu-temp.sh' ]

curves:
  - id: gpu_curve
    # The type of the curve, one of: linear | function
    linear:
      # The sensor ID to use as a temperature input
      sensor: cmd_max_gpu
      # Sensor input value at which the curve is at minimum speed
      min: 30
      # Sensor input value at which the curve is at maximum speed
      max: 75
      
statistics:
  # Whether to enable the prometheus exporter or not
  enabled: true
  # The port to expose the exporter on
  port: 9000
  
runFanInitializationInParallel: false

max-gpu-temp.sh

#!/bin/bash

echo $(( $(/usr/bin/nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader | sort -g | tail -n 1) * 1000 ))

[markusressel]

Hmm I cannot see anything obviously wrong there.
What log output do you get when you run the daemon with --verbose?

Should look something like this:

INFO: Using configuration file at: ./fan2go.yaml
INFO: Starting statistics server...
INFO: Starting REST server...
INFO: Gathering sensor data for cpu_fan...
⇨ http server started on 127.0.0.1:9001
INFO: Loading fan curve data for fan 'cpu_fan'...
INFO: PWM settings of fan 'cpu_fan': Min 0, Start 1, Max 255
INFO: Starting controller loop for fan 'cpu_fan'

[mcgrizzz]

This is what I get when running fan2go with the verbose flag.

 INFO  Using configuration file at: /etc/fan2go/fan2go.yaml
 INFO  Starting statistics server...
 INFO  Gathering sensor data for BOTTOM_MIDDLE...
 INFO  Gathering sensor data for BOTTOM_LEFT...
 INFO  Gathering sensor data for FRONT_TOP...
 INFO  Gathering sensor data for CPU_MIDDLE...
 INFO  Gathering sensor data for CPU_FRONT...
 INFO  Gathering sensor data for BOTTOM_RIGHT...
 INFO  Gathering sensor data for FRONT_BOTTOM...
 INFO  Loading fan curve data for fan 'BOTTOM_LEFT'...
 INFO  Loading fan curve data for fan 'FRONT_BOTTOM'...
 INFO  Loading fan curve data for fan 'CPU_FRONT'...
 INFO  Loading fan curve data for fan 'CPU_MIDDLE'...
 INFO  Loading fan curve data for fan 'BOTTOM_MIDDLE'...
 INFO  Loading fan curve data for fan 'BOTTOM_RIGHT'...
 INFO  Loading fan curve data for fan 'FRONT_TOP'...
 INFO  PWM settings of fan 'BOTTOM_LEFT': Min 0, Start 0, Max 244
 INFO  Starting controller loop for fan 'BOTTOM_LEFT'
 INFO  PWM settings of fan 'BOTTOM_RIGHT': Min 0, Start 0, Max 238
 INFO  Starting controller loop for fan 'BOTTOM_RIGHT'
 INFO  PWM settings of fan 'CPU_MIDDLE': Min 0, Start 0, Max 253
 INFO  Starting controller loop for fan 'CPU_MIDDLE'
 INFO  PWM settings of fan 'FRONT_BOTTOM': Min 0, Start 0, Max 243
 INFO  Starting controller loop for fan 'FRONT_BOTTOM'
 INFO  PWM settings of fan 'BOTTOM_MIDDLE': Min 0, Start 0, Max 238
 INFO  Starting controller loop for fan 'BOTTOM_MIDDLE'
 INFO  PWM settings of fan 'FRONT_TOP': Min 0, Start 0, Max 250
 INFO  Starting controller loop for fan 'FRONT_TOP'
 INFO  PWM settings of fan 'CPU_FRONT': Min 0, Start 0, Max 250

[markusressel]

I have restructured the webserver code quite a bit, please give it a new try with the latest version.

[mcgrizzz]

I've tried out the newest version. It now serves the metrics endpoint correctly. It does, however, take around 30-40 seconds through the browser and around 10 seconds through curl to receive a response.

Is this expected?

[markusressel]

Well thats some progress, but no, a 30 second delay is not expeted.
On my machine the response is delivered within a couple milliseconds.

[markusressel]

I will go ahead and close this for now since I cannot reproduce the issue. If you find any new infos regarding this that need a fix in fan2go, feel free to post or create a new issue 🤓