Cleanups to help fix CI

[nshalman]

Description

Fix CI

Why is this needed

CI is broken

How Has This Been Tested?

• CI

How are existing users impacted? What migration steps/scripts do we need?

Checklist:

I have:

• updated the documentation and/or roadmap (if required)
• added unit or e2e tests
• provided instructions on how to upgrade

[codecov]

Codecov Report

Merging #72 (79a7fb6) into main (d83a28e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #72   +/-   ##
=======================================
  Coverage   33.89%   33.89%           
=======================================
  Files           3        3           
  Lines         413      413           
=======================================
  Hits          140      140           
  Misses        250      250           
  Partials       23       23           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d83a28e...79a7fb6. Read the comment docs.

[nshalman]

I'm concerned that CI is working within PRs but broken on main, if that's the case, landing this will be a nice improvement, but won't fix the build for pushing to quay and we'll still need deeper debugging and a follow-up.

[markjacksonfishing]

Please continue with that deeper dive, this is a priority.

[nshalman]

Okay, I've managed to trip the intermittent test failure:

=== RUN   TestServe/serve_endpoint_success
{"level":"info","ts":1640096102.091404,"caller":"http-server/http_server.go:65","msg":"Starting http server","service":"github.com/tinkerbell/hegel","pkg":"httpserver","port":52000}
    http_server_test.go:934: request failed: Get "http://localhost:52000/_packet/version": dial tcp [::1]:52000: connect: connection refused
=== RUN   TestServe/serve_endpoint_failed
--- FAIL: TestServe (0.00s)
    --- FAIL: TestServe/serve_endpoint_success (0.00s)
    --- PASS: TestServe/serve_endpoint_failed (0.00s)
FAIL
FAIL	github.com/tinkerbell/hegel/http-server	0.025s
?   	github.com/tinkerbell/hegel/metrics	[no test files]
?   	github.com/tinkerbell/hegel/xff	[no test files]
FAIL

Flaky tests are the worst. This might require deeper changes to fix that test.

[nshalman]

The issue is the use of a hardcoded port

hegel/http-server/http_server_test.go

Lines 914 to 915 in 9bb4194

	mport := 52000
	metricsPort = &mport

being assigned to a global variable

hegel/http-server/http_server.go

Line 26 in 9bb4194

metricsPort = flag.Int("http_port", env.Int("HEGEL_HTTP_PORT", 50061), "Port to listen on http")

This is exacerbated by the fact that during CI we run tests twice:

hegel/.github/workflows/ci.yaml

Lines 24 to 27 in 9bb4194

	- name: go test
	run: go test -v ./...
	- name: go test coverage
	run: go test -coverprofile=coverage.txt ./...

This greatly increases the chances that the port won't be properly available. For now the easy fix is to consolidate to a single test run in CI, and later a deeper refactor can make this code more flexible about how it chooses a port for running tests.