Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bind addresses are used as ip addresses to connect to server #10381

Closed
Tracked by #10344
wkloucek opened this issue Oct 22, 2024 · 11 comments
Closed
Tracked by #10344

Bind addresses are used as ip addresses to connect to server #10381

wkloucek opened this issue Oct 22, 2024 · 11 comments
Assignees
Labels

Comments

@wkloucek
Copy link
Contributor

Describe the bug

The healthchecks use the http server bind address for connection to the server, eg:

healthHandlerConfiguration := handlers.NewCheckHandlerConfiguration().
WithLogger(options.Logger).
WithCheck("web reachability", checks.NewHTTPCheck(options.Config.HTTP.Addr))

This fails if the http server bind address uses a bind-all-interfaces logic, like 0.0.0.0.

Steps to reproduce

  1. Deploy eg. the development deployment example in Kubernetes
  2. look at the pods

Expected behavior

Healthchecks succeed -> no pod restarts

Actual behavior

Healthchecks fail -> pod restarts

Setup

oCIS 6.6.0 in Kubernetes, eg. owncloud/ocis-charts#778

Additional context

@wkloucek
Copy link
Contributor Author

wkloucek commented Oct 22, 2024

I tried to workaround this by replacing the wildcard bind 0.0.0.0 by an actual interface ip, eg:

            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP

            - name: GATEWAY_GRPC_ADDR
              value: $(POD_IP):9142
            - name: GATEWAY_DEBUG_ADDR
              value: $(POD_IP):9143

But this fails on the gateway, because it tries to talk to itself via 127.0.0.1. During a quick search, I didn't find a config option to tell the gateway where it could contact itself...

@dj4oC dj4oC moved this from Qualification to Prio 1 in Infinite Scale Team Board Oct 23, 2024
@dj4oC
Copy link
Contributor

dj4oC commented Oct 23, 2024

priority escalation since it is blocking us

@wkloucek wkloucek changed the title Bind addressees are used as ip addresses to connect to server Bind addresses are used as ip addresses to connect to server Oct 23, 2024
@micbar
Copy link
Contributor

micbar commented Oct 23, 2024

Known issue since yesterday, working on it. We are holding the release candidate until that is fixed.

@micbar
Copy link
Contributor

micbar commented Oct 23, 2024

Fixed by #10390 #10392 #10394

@micbar micbar closed this as completed Oct 23, 2024
@github-project-automation github-project-automation bot moved this from Prio 1 to Done in Infinite Scale Team Board Oct 23, 2024
@micbar
Copy link
Contributor

micbar commented Oct 23, 2024

Release 6.6.1 in preparation #10401

@wkloucek
Copy link
Contributor Author

Still an issue on v6.6.1-rc.1

@wkloucek wkloucek reopened this Oct 24, 2024
@github-project-automation github-project-automation bot moved this from Done to In progress in Infinite Scale Team Board Oct 24, 2024
@wkloucek
Copy link
Contributor Author

Fixed by #10390 #10392 #10394

I'm not sure how the port changes will affect this bug. The config changes have actually no effect for the Helm Chart because this is setting this already since forever.

The difference / problem is, that oCIS by default has a bind address 127.0.0.1, which turns out to be a valid IP address, too. The Helm Chart uses the bind address 0.0.0.0 (speak: all interfaces), which is NO valid ip address.

Therefore the health checks fail in many pods, because they try to use the bind address to do the checks...

@dragonchaser
Copy link
Contributor

I was not aware of that, sorry. We have to figure out the ip of one of the interface the bind actually happens and check with that.

@wkloucek
Copy link
Contributor Author

We have to figure out the ip of one of the interface the bind actually happens and check with that.

I tried to do that on the Kubernetes side as a workaround, but failed on another place: #10381 (comment)

@dragonchaser
Copy link
Contributor

#10405 might further improve things...

@wkloucek
Copy link
Contributor Author

wkloucek commented Oct 24, 2024

Seems to be fixed this time! I have no pod restarts when waiting for 5 minutes. (Testing with oCIS 6.6.1-rc.2)

@github-project-automation github-project-automation bot moved this from In progress to Done in Infinite Scale Team Board Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests

4 participants