Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn off host resolution #50

Open
plasorak opened this issue Mar 13, 2023 · 4 comments
Open

Turn off host resolution #50

plasorak opened this issue Mar 13, 2023 · 4 comments

Comments

@plasorak
Copy link
Contributor

This line assumes that the C++ should resolve the IP address of the host. This breaks DAQ applications running in K8s

I think the code here is superseded by the connectivity service which is where we should resolve all the IPs, right?

My two cents here would be to remove this line when we have the connectivity service running in a "consolidated way."

@eflumerf eflumerf transferred this issue from DUNE-DAQ/ipm Mar 13, 2023
@eflumerf
Copy link
Member

Transferring to iomanager, as I think the bulk of the changes will have to be made here. ZMQ does not support hostnames in its URIs, only IPs, so some resolution must be done. However, the Connectivity Service should be given the versions of the URI that contain the hostname so that the resolution is done correctly within the K8s containers.

@plasorak
Copy link
Contributor Author

We checked with Tiago that passing hostname here works inside K8s (I'm assuming that the IP/hostnames are the same in K8s or that something else does it).
So basically what I understand should happen is that the boot.use_k8s should be translated to no a turn_off_resolution iomanager parameter around and make this possible. I'll have a look at this next week with Tiago.

@eflumerf
Copy link
Member

eflumerf commented May 3, 2023

So more research into ZMQ shows that I was partially incorrect...calls to bind require IP addresses, but calls to connect can use a hostname. This led to my current set of changes, where IOManager does not update its connection strings with resolved information by default, instead publishing to the Connectivity Service the unresolved hostnames. The exception is when a wildcard is present in the input connection string, either a port (e.g. tcp://dfo:*), which will be allocated to an epheremal port on bind, or a host (e.g. tcp://*:1234), which listens on all interfaces, but should be given to Connectivity Service as one of the IPs that the socket was actually bound to. (I believe that this functionality is never used in DUNE-DAQ.)

@plasorak
Copy link
Contributor Author

plasorak commented May 3, 2023

So, just trying to hammer this home:

  • In cases where both SSH and connectivity service are used: daqconf here and here will create a connection of the form tcp://{dfo}:* in the conf json. This is then passed to nanorc which converts it to tcp://np04-srv-001:*. This then gets published in the connectivity services as tcp://np04-srv-001:40224. The addresses given to ZMQ are tcp://10.73.136.56:* and tcp://np04-srv-001:40224 for the receiver and sender, respectively.

  • In cases where SSH and no connectivity service are used: the same piece of code in daqconf will create a connection of the form tcp://{dfo}:1234. Nanorc resolves the hostname tcp://np04-srv-001:1234. The addresses given to ZMQ are tcp://10.73.136.56:12353 and tcp://np04-srv-004:12352 for the receiver and sender, respectively

  • In cases where K8s and connectivity service are used: the connection is tcp://dfo:1234 in the conf json. Nanorc doesn’t do anything to this. The published connectivity services addresses are of the form tcp://dfo:1234. The addresses given to ZMQ are tcp://10.244.8.115:1234 and tcp://dfo:1234 for the receiver and sender, respectively. I’m a bit surprised that the receiver side works.

  • I haven't investigated the no k8s no connectivity service.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants