Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding TLS to node exporter. Server Auth only. #1198

Closed
wants to merge 54 commits into from
Closed
Changes from 37 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
b188e1a
Added TLS listener to node exporter. Server auth only. Pass in cert a…
achiuBAE Dec 12, 2018
7bc2aae
html head tag indent corrected back to normal. no code change
achiuBAE Dec 12, 2018
288bec4
Certificate rotation added requires testing
ksherryBAE Jan 11, 2019
47c0c3b
Certificates reloaded on intial ClientHello
ksherryBAE Jan 31, 2019
65a3a7f
Fixed Spelling
ksherryBAE Jan 31, 2019
fe02aa3
Merge branch 'master' of https://github.com/prometheus/node_exporter …
ksherryBAE Jan 31, 2019
7cdd689
minor formatting
ksherryBAE Jan 31, 2019
379cc25
https package created code duplicated and utilised cleanup required
ksherryBAE Jan 31, 2019
4e1d10a
Spelling fix
ksherryBAE Feb 1, 2019
9df2228
Merge branch 'master' of https://github.com/prometheus/node_exporter …
ksherryBAE Feb 6, 2019
4a8ed90
Minor code formatting
ksherryBAE Feb 6, 2019
3785913
Minor code formatting for CI
ksherryBAE Feb 6, 2019
d5a9dba
Minor code formatting for CI - imports properly ordered
ksherryBAE Feb 6, 2019
96c912b
Struct restructuring
ksherryBAE Feb 6, 2019
a878951
Merge branch 'tls-server-auth' into tls-server-auth-package
ksherryBAE Feb 7, 2019
e62d1a5
Struct adjustments
ksherryBAE Feb 7, 2019
d91ddcb
Merge branch 'tls-server-auth-package' of https://github.com/BAESyste…
ksherryBAE Feb 7, 2019
f232e11
Config extracted out to https package
ksherryBAE Feb 7, 2019
57ff43f
Unecessary struct removed
ksherryBAE Feb 7, 2019
1e2777c
Files cleaned up extraction of server creation started
ksherryBAE Feb 15, 2019
9be5023
Reset tlsConfig to defaults
ksherryBAE Feb 15, 2019
8bf8714
Reset tlsConfig to defaults code formatted
ksherryBAE Feb 15, 2019
0552afe
Packages added
ksherryBAE Feb 21, 2019
b844599
removed go. files
ksherryBAE Feb 21, 2019
5fb1cda
moved GetCertificate into GetTLSConfig
jritchieBAE Feb 21, 2019
9c3e40a
Yaml struct importing
ksherryBAE Feb 25, 2019
9b533b2
-mod merged in
ksherryBAE Feb 25, 2019
186ca13
Functional Yaml Reading
ksherryBAE Feb 25, 2019
0e5792c
Config yaml without CA's added
ksherryBAE Feb 25, 2019
c28ef3c
Cert and Key moved from flags to config
ksherryBAE Feb 26, 2019
55f9b51
CA pools added to config
ksherryBAE Feb 26, 2019
eafb071
Removed conntrack from vendor
ksherryBAE Feb 26, 2019
e7e360b
tls flag renamed tls-config
ksherryBAE Feb 26, 2019
ed87bca
minor formatting
ksherryBAE Feb 26, 2019
55c2e45
Remove local file path references
ksherryBAE Feb 26, 2019
51cbd1e
updated git ignore to ignore yaml and yml extensions
ksherryBAE Feb 26, 2019
632fca9
renamed config base
ksherryBAE Feb 26, 2019
440d75f
Re-added yaml&yml to gitignore
ksherryBAE Feb 26, 2019
cf6a026
tls.yml removed from /node_exporter
ksherryBAE Feb 26, 2019
7d05f7e
ran gofmt
jritchieBAE Feb 27, 2019
b49f96e
log.Fatalf now log.Fatal
jritchieBAE Feb 27, 2019
2fc0145
renamed so not exported from module
jritchieBAE Feb 27, 2019
4bb0273
BuildNameToCertificate moved to certificate init
jritchieBAE Feb 27, 2019
17b13f4
moved declaration of tls.Config{} into loadConfigFromYaml
jritchieBAE Feb 27, 2019
186e533
server declaration to align listen calls
jritchieBAE Feb 27, 2019
1f28e2a
separate listen calls combined into convenience method https.Listen(*…
jritchieBAE Feb 27, 2019
95c3aa3
dependencies added to go.sum
jritchieBAE Feb 27, 2019
4109175
renamed struct so it can be exported
jritchieBAE Feb 27, 2019
f3f5d3f
split function into two to allow code to be reused
jritchieBAE Feb 27, 2019
8b34555
TLS extracted to https package with config file for options
ksherryBAE Feb 27, 2019
baa0545
Spelling and removal of redundant bool comparisons, tidy up of spelli…
ksherryBAE Feb 27, 2019
ed5005f
Merge remote-tracking branch 'prometheus/master' into tls-server-auth
ksherryBAE Mar 6, 2019
45812d4
Update README.md
ksherryBAE Mar 7, 2019
959b61f
Update README.md
ksherryBAE Mar 7, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -28,7 +28,7 @@ jobs:
steps:
- checkout
- run: sudo pip install codespell
- run: codespell --skip=".git,./vendor,ttar,go.mod,go.sum" -L uint
- run: codespell --skip=".git,./vendor,ttar" -L uint

build:
machine: true
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -3,6 +3,10 @@
*.a
*.so

# Config files
*.yml
*.yaml

# Folders
_obj
_test
16 changes: 1 addition & 15 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -2,33 +2,19 @@

### **Breaking changes**

* Renamed `interface` label to `device` in netclass collector for consistency with
other network metrics #1224
* The cpufreq metrics now separate the `cpufreq` and `scaling` data based on what the driver provides. #1248
* The labels for the network_up metric have changed, see issue #1236
* Bonding collector now uses `mii_status` instead of `operstatus` #1124
* Several systemd metrics have been turned off by default to improve performance #1254
These include unit_tasks_current, unit_tasks_max, service_restart_total, and unit_start_time_seconds
* The systemd collector blacklist now includes automount, device, mount, and slice units by default. #1255
The cpufreq metrics now separate the `cpufreq` and `scaling` data based on what the driver provides. #1248

### Changes

* [BUGFIX]
* [CHANGE] Renamed `interface` label to `device` in netclass collector #1224
* [BUGFIX] Add fallback for missing /proc/1/mounts #1172
* [CHANGE] Add TCPSynRetrans to netstat default filter #1143
* [CHANGE] Add a limit to the number of in-flight requests #1166
* [CHANGE] Add separate cpufreq and scaling metrics #1248
* [CHANGE] Several systemd metrics have been turned off by default to improve performance #1254
* [CHANGE] Expand systemd collector blacklist #1255
* [CHANGE] Split cpufreq metrics into a separate collector #1253
* [ENHANCEMENT] Add Infiniband counters #1120
* [ENHANCEMENT] Move network_up labels into new metric network_info #1236
* [FEATURE] Add a flag to disable exporter metrics #1148
* [FEATURE] Add kstat-based Solaris metrics for boottime, cpu and zfs collectors #1197
* [FEATURE] Add uname collector for FreeBSD #1239
* [FEATURE] Add diskstats collector for OpenBSD #1250
* [CHANGE] Bonding state uses mii_status #1124

## 0.17.0 / 2018-11-30

5 changes: 3 additions & 2 deletions Makefile.common
Original file line number Diff line number Diff line change
@@ -36,8 +36,7 @@ GO_VERSION ?= $(shell $(GO) version)
GO_VERSION_NUMBER ?= $(word 3, $(GO_VERSION))
PRE_GO_111 ?= $(shell echo $(GO_VERSION_NUMBER) | grep -E 'go1\.(10|[0-9])\.')

GOVENDOR :=
GO111MODULE :=
unexport GOVENDOR
ifeq (, $(PRE_GO_111))
ifneq (,$(wildcard go.mod))
# Enforce Go modules support just in case the directory is inside GOPATH (and for Travis CI).
@@ -58,6 +57,8 @@ $(warning Some recipes may not work as expected as the current Go runtime is '$(
# This repository isn't using Go modules (yet).
GOVENDOR := $(FIRST_GOPATH)/bin/govendor
endif

unexport GO111MODULE
endif
PROMU := $(FIRST_GOPATH)/bin/promu
STATICCHECK := $(FIRST_GOPATH)/bin/staticcheck
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -32,8 +32,7 @@ bonding | Exposes the number of configured and active slaves of Linux bonding in
boottime | Exposes system boot time derived from the `kern.boottime` sysctl. | Darwin, Dragonfly, FreeBSD, NetBSD, OpenBSD, Solaris
conntrack | Shows conntrack statistics (does nothing if no `/proc/sys/net/netfilter/` present). | Linux
cpu | Exposes CPU statistics | Darwin, Dragonfly, FreeBSD, Linux, Solaris
cpufreq | Exposes CPU frequency statistics | Linux, Solaris
diskstats | Exposes disk I/O statistics. | Darwin, Linux, OpenBSD
diskstats | Exposes disk I/O statistics. | Darwin, Linux
edac | Exposes error detection and correction statistics. | Linux
entropy | Exposes available entropy. | Linux
exec | Exposes execution statistics. | Dragonfly, FreeBSD
4 changes: 2 additions & 2 deletions collector/bonding_linux.go
Original file line number Diff line number Diff line change
@@ -82,10 +82,10 @@ func readBondingStats(root string) (status map[string][2]int, err error) {
}
sstat := [2]int{0, 0}
for _, slave := range strings.Fields(string(slaves)) {
state, err := ioutil.ReadFile(filepath.Join(root, master, fmt.Sprintf("lower_%s", slave), "bonding_slave", "mii_status"))
state, err := ioutil.ReadFile(filepath.Join(root, master, fmt.Sprintf("lower_%s", slave), "operstate"))
if os.IsNotExist(err) {
// some older? kernels use slave_ prefix
state, err = ioutil.ReadFile(filepath.Join(root, master, fmt.Sprintf("slave_%s", slave), "bonding_slave", "mii_status"))
state, err = ioutil.ReadFile(filepath.Join(root, master, fmt.Sprintf("slave_%s", slave), "operstate"))
}
if err != nil {
return nil, err
107 changes: 107 additions & 0 deletions collector/cpu_linux.go
Original file line number Diff line number Diff line change
@@ -23,11 +23,18 @@ import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/common/log"
"github.com/prometheus/procfs"
"github.com/prometheus/procfs/sysfs"
)

type cpuCollector struct {
cpu *prometheus.Desc
cpuGuest *prometheus.Desc
cpuFreq *prometheus.Desc
cpuFreqMin *prometheus.Desc
cpuFreqMax *prometheus.Desc
scalingFreq *prometheus.Desc
scalingFreqMin *prometheus.Desc
scalingFreqMax *prometheus.Desc
cpuCoreThrottle *prometheus.Desc
cpuPackageThrottle *prometheus.Desc
}
@@ -45,6 +52,36 @@ func NewCPUCollector() (Collector, error) {
"Seconds the cpus spent in guests (VMs) for each mode.",
[]string{"cpu", "mode"}, nil,
),
cpuFreq: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "frequency_hertz"),
"Current cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
cpuFreqMin: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "frequency_min_hertz"),
"Minimum cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
cpuFreqMax: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "frequency_max_hertz"),
"Maximum cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
scalingFreq: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "scaling_frequency_hertz"),
"Current scaled cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
scalingFreqMin: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "scaling_frequency_min_hrts"),
"Minimum scaled cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
scalingFreqMax: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "scaling_frequency_max_hrts"),
"Maximum scaled cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
cpuCoreThrottle: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "core_throttles_total"),
"Number of times this cpu core has been throttled.",
@@ -63,12 +100,82 @@ func (c *cpuCollector) Update(ch chan<- prometheus.Metric) error {
if err := c.updateStat(ch); err != nil {
return err
}
if err := c.updateCPUfreq(ch); err != nil {
return err
}
if err := c.updateThermalThrottle(ch); err != nil {
return err
}
return nil
}

// updateCPUfreq reads /sys/devices/system/cpu/cpu* and expose cpu frequency statistics.
func (c *cpuCollector) updateCPUfreq(ch chan<- prometheus.Metric) error {
fs, err := sysfs.NewFS(*sysPath)
if err != nil {
return fmt.Errorf("failed to open sysfs: %v", err)
}

cpuFreqs, err := fs.NewSystemCpufreq()
if err != nil {
return err
}

// sysfs cpufreq values are kHz, thus multiply by 1000 to export base units (hz).
// See https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt
for _, stats := range cpuFreqs {
if stats.CpuinfoCurrentFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.cpuFreq,
prometheus.GaugeValue,
float64(*stats.CpuinfoCurrentFrequency)*1000.0,
stats.Name,
)
}
if stats.CpuinfoMinimumFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.cpuFreqMin,
prometheus.GaugeValue,
float64(*stats.CpuinfoMinimumFrequency)*1000.0,
stats.Name,
)
}
if stats.CpuinfoMaximumFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.cpuFreqMax,
prometheus.GaugeValue,
float64(*stats.CpuinfoMaximumFrequency)*1000.0,
stats.Name,
)
}
if stats.ScalingCurrentFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.scalingFreq,
prometheus.GaugeValue,
float64(*stats.ScalingCurrentFrequency)*1000.0,
stats.Name,
)
}
if stats.ScalingMinimumFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.scalingFreqMin,
prometheus.GaugeValue,
float64(*stats.ScalingMinimumFrequency)*1000.0,
stats.Name,
)
}
if stats.ScalingMaximumFrequency != nil {
ch <- prometheus.MustNewConstMetric(
c.scalingFreqMax,
prometheus.GaugeValue,
float64(*stats.ScalingMaximumFrequency)*1000.0,
stats.Name,
)
}
}
return nil
}

// updateThermalThrottle reads /sys/devices/system/cpu/cpu* and expose thermal throttle statistics.
func (c *cpuCollector) updateThermalThrottle(ch chan<- prometheus.Metric) error {
cpus, err := filepath.Glob(sysFilePath("devices/system/cpu/cpu[0-9]*"))
70 changes: 68 additions & 2 deletions collector/cpu_solaris.go
Original file line number Diff line number Diff line change
@@ -17,17 +17,20 @@
package collector

import (
"fmt"
"strconv"

"github.com/prometheus/client_golang/prometheus"
kstat "github.com/siebenmann/go-kstat"
"github.com/siebenmann/go-kstat"
)

// #include <unistd.h>
import "C"

type cpuCollector struct {
cpu typedDesc
cpu typedDesc
cpuFreq *prometheus.Desc
cpuFreqMax *prometheus.Desc
}

func init() {
@@ -37,10 +40,30 @@ func init() {
func NewCpuCollector() (Collector, error) {
return &cpuCollector{
cpu: typedDesc{nodeCPUSecondsDesc, prometheus.CounterValue},
cpuFreq: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "frequency_hertz"),
"Current cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
cpuFreqMax: prometheus.NewDesc(
prometheus.BuildFQName(namespace, cpuCollectorSubsystem, "frequency_max_hertz"),
"Maximum cpu thread frequency in hertz.",
[]string{"cpu"}, nil,
),
}, nil
}

func (c *cpuCollector) Update(ch chan<- prometheus.Metric) error {
if err := c.updateCPUstats(ch); err != nil {
return err
}
if err := c.updateCPUfreq(ch); err != nil {
return err
}
return nil
}

func (c *cpuCollector) updateCPUstats(ch chan<- prometheus.Metric) error {
ncpus := C.sysconf(C._SC_NPROCESSORS_ONLN)

tok, err := kstat.Open()
@@ -72,3 +95,46 @@ func (c *cpuCollector) Update(ch chan<- prometheus.Metric) error {
}
return nil
}

func (c *cpuCollector) updateCPUfreq(ch chan<- prometheus.Metric) error {
ncpus := C.sysconf(C._SC_NPROCESSORS_ONLN)

tok, err := kstat.Open()
if err != nil {
return err
}

defer tok.Close()

for cpu := 0; cpu < int(ncpus); cpu++ {
ksCPUInfo, err := tok.Lookup("cpu_info", cpu, fmt.Sprintf("cpu_info%d", cpu))
if err != nil {
return err
}
cpuFreqV, err := ksCPUInfo.GetNamed("current_clock_Hz")
if err != nil {
return err
}

cpuFreqMaxV, err := ksCPUInfo.GetNamed("clock_MHz")
if err != nil {
return err
}

lcpu := strconv.Itoa(cpu)
ch <- prometheus.MustNewConstMetric(
c.cpuFreq,
prometheus.GaugeValue,
float64(cpuFreqV.UintVal),
lcpu,
)
// Multiply by 1e+6 to convert MHz to Hz.
ch <- prometheus.MustNewConstMetric(
c.cpuFreqMax,
prometheus.GaugeValue,
float64(cpuFreqMaxV.IntVal)*1e+6,
lcpu,
)
}
return nil
}
Loading