net: dns: dns-sd: support dns service discovery

[cfriedt]

This change adds support for DNS Service Discovery (DNS-SD) as described in RFC 6763. It also adds DNS-SD support to the mdns_responder service and sample application as well as a function call net_context_port_in_use().

Fixes #29099
Fixes #29429
Fixes #29649

[cfriedt]

WIP. I have two patch sets - one where I unit-tested it in Zephyr, and another where I unit tested using gtest. In the end, the gtest method allowed me to make faster progress, but the price I pay now is that I need to transpose those tests from gtest to zephyr.

Results are promising though.

[jukkar]

Some initial notes.

[cfriedt]

Just "transposed" my test suite over from gtest, which had 100% coverage. There are likely still some issues to sort out here (e.g. currently there are pointer comparisons where there were previously vector comparisons).

[cfriedt]

Close - running into random faults, but otherwise, I'm able to get responses:

$ avahi-browse -t -r _zephyr._tcp
+ lowpan0 IPv6 Zephyr RTOS                                   _zephyr._tcp         local
= lowpan0 IPv6 Zephyr RTOS                                   _zephyr._tcp         local
   hostname = [Zephyr\032RTOS.local]
   address = [2001:db8::1]
   port = [4242]
   txt = []

There is a similar UDP service that can be queried as well. I'm currently testing with the cc1352r1_launchxl, but will test shortly with nrf52840dk_nrf52840.

It can be tested with the mdns_responder sample app.

[jukkar]

I think you could add here the changes you have to mdns_responder sample, that way it is easier to see how the thing is being used.
One thing I noticed, that we should do also changes to socket layer as we should only start to advertise the service after there is a local socket bound to the said service. So the application etc can register a service using the macros, but it should be only enabled at runtime after there is really a service running.

[cfriedt]

Currently there is a stack overflow on mps2_an385 in the main stack. Going to see if increasing it to 2048 fixes the issue.

[cfriedt]

I think you could add here the changes you have to mdns_responder sample, that way it is easier to see how the thing is being used.

I was thinking about creating a separate ticket for that but just including it here as a PR since they are pretty tightly coupled; do we have a non-multicast DNS service as well?

One thing I noticed, that we should do also changes to socket layer as we should only start to advertise the service after there is a local socket bound to the said service. So the application etc can register a service

Hmm... would make sense. I also considered adding some callbacks to the REGISTER() macro. what do you think of

• is_ready()
• handle_error()
• on_query()

Could possibly use flags instead of individual functions too.

That might actually facilitate dynamic TXT 😳

[jukkar]

I was thinking about creating a separate ticket for that but just including it here as a PR since they are pretty tightly coupled; do we have a non-multicast DNS service as well?

We could introduce DNS-SD to other samples too if they enable DNS. For example echo-server is not enabling DNS by default, but we could have an overlay there which could enable DNS-SD so that the app can start to advertise the service.

[jukkar]

Hmm... would make sense. I also considered adding some callbacks to the REGISTER() macro. what do you think of

* is_ready()
* handle_error()
* on_query()

Could possibly use flags instead of individual functions too.

That might actually facilitate dynamic TXT flushed

Callbacks are not really possible if the application is running in usermode as the kernel mode component cannot pass any data in a safe way to the application. If possible, lets try to avoid callbacks unless really mandatory.

In #29200 I did a bit similar functionality. In that PR, when application binds to a local address, the code in net_context_bind() will call a function that "reserves" the used IP address in the network interface. This way we can keep the IP address around in the network interface if there are any users for it. And when the socket is closed, we "release" the used IP address.

So for DNS-SD, we could do it similar way. If application has registered DNS-SD service (using the macro, we could allow runtime configuration in the future too, this was what I suggested earlier), the bind code will call some dns-sd API function to let it know that a local port was bound, dns-sd could then verify that this port is indeed an already registered service and enable advertisements. When the said socket is closed, the advertisements are disabled.

[cfriedt]

We could introduce DNS-SD to other samples too if they enable DNS. For example echo-server is not enabling DNS by default, but we could have an overlay there which could enable DNS-SD so that the app can start to advertise the service.

That might be a lot of pressure on one ticket - can we limit this one to "support" and then "enable" in samples later?

[jukkar]

That might be a lot of pressure on one ticket - can we limit this one to "support" and then "enable" in samples later?

The problem with this "early" support is that it is not possible to see example how the DNS-SD would work if we not have sample that uses it properly (with also changes in socket bind()). I am hesitant to have code merged if the usage of it is not clear.

[jukkar]

The problem with this "early" support is that it is not possible to see example how the DNS-SD would work if we not have sample that uses it properly (with also changes in socket bind()). I am hesitant to have code merged if the usage of it is not clear.

That said, we could have another PR that adds sample etc if you do not want to add extra commits to this one.

[cfriedt]

That might be a lot of pressure on one ticket - can we limit this one to "support" and then "enable" in samples later?

The problem with this "early" support is that it is not possible to see example how the DNS-SD would work if we not have sample that uses it properly (with also changes in socket bind()). I am hesitant to have code merged if the usage of it is not clear.

Sorry for the confusion - I will definitely add DNS-SD to the mDNS responder application, I just mean that it might be a bit much to add DNS-SD support to every networked sample application.

As long as we can limit the sample changes to mDNS responder, that is ok with me. You?

[jukkar]

I just mean that it might be a bit much to add DNS-SD support to every networked sample application.

As long as we can limit the sample changes to mDNS responder, that is ok with me. You?

Agreed, I did not mean that we need to add DNS-SD support to every network samp app either.
Adding it to mdns responder is a good start and I am ok with it.

[cfriedt]

In #29200 I did a bit similar functionality. In that PR, when application binds to a local address, the code in net_context_bind() will call a function that "reserves" the used IP address in the network interface. This way we can keep the IP address around in the network interface if there are any users for it. And when the socket is closed, we "release" the used IP address.

So for DNS-SD, we could do it similar way. If application has registered DNS-SD service (using the macro, we could allow runtime configuration in the future too, this was what I suggested earlier), the bind code will call some dns-sd API function to let it know that a local port was bound, dns-sd could then verify that this port is indeed an already registered service and enable advertisements. When the said socket is closed, the advertisements are disabled.

Since all of the DNS-SD records are const, it might be better to query the network subsystem from DNS_SD to see whether or not any sockets are open on a given address / port. I mean, a simple way to do it would be to try and open a socket / port and immediately close it afterward, but that gets a bit racey.

I like the idea though.

Another item that ties into this idea is the simple fact that DNS-SD was designed so that ports did not need to be preassigned. A service should be able to bind to 0 and get an ephemeral port number and have DNS-SD advertise that randomly assigned port number as easily as if it were preassigned.

In particular, I think that would make for a good "next ticket" item, as I think it would solve both of our concerns, @jukkar. E.g. if the port number is 0, that means the service has not yet started and DNS-SD should not advertise it.

However, using a uint16_t would break because we desire const-ness. So, make it a const pointer. If the port number were a pointer, then it's possible that user code would need to be granted special permissions to access it (i.e. to set it after a call to bind(2)), and that struct dns_sd_rec would need to be a struct k_object. @andrewboie, does that make sense?

[cfriedt]

OK - I think I'm basically done now, as long as everything is green. Please have a look over at your earliest convenience.

[cfriedt]

@jukkar, I believe that the last commit should address your change request.

Changes were implemented

[jukkar]

Since all of the DNS-SD records are const, it might be better to query the network subsystem from DNS_SD to see whether or not any sockets are open on a given address / port. I mean, a simple way to do it would be to try and open a socket / port and immediately close it afterward, but that gets a bit racey.

I do not think we need to do those kind of kludges like open a socket and then immediately close it. We can just implement a function (could be kept private) in socket lib, that checks whether a given port is opened or not, and then call that from DNS_SD to check whether the service should be advertised or not.

[jukkar]

Had I been able to, I would have just included the .c file in the test.

Including the .c file could be a nice solution here, what was the issue preventing that?

[cfriedt]

Since all of the DNS-SD records are const, it might be better to query the network subsystem from DNS_SD to see whether or not any sockets are open on a given address / port. I mean, a simple way to do it would be to try and open a socket / port and immediately close it afterward, but that gets a bit racey.

I do not think we need to do those kind of kludges like open a socket and then immediately close it.

Same - I was just using it as a bad example.

We can just implement a function (could be kept private) in socket lib, that checks whether a given port is opened or not, and then call that from DNS_SD to check whether the service should be advertised or not.

Should the function be added in this PR?

[jukkar]

Should the function be added in this PR?

I am fine with that, in a separate commit of course.

[cfriedt]

Update - added ephemeral port support in this PR as well. So that would allow a service to just use any available port that was returned by bind(2) and not necessarily a fixed one (but fixed ports are also fine).

[cfriedt]

I think I will need to create a small network service in mdns_responder and bind it to a port, because obviously with the additional change where it only advertises bound services, mdns_responder no longer responds to PTR queries.

[rlubos]

LGTM, pretty neat 👍

[cfriedt]

Some last minute changes - sorry.

As it turns out, the local_port argument to check_used_port() in net_context.c is actually in network byte order. @jukkar - if you still would prefer to have that in host byte order, then I would suggest making another issue for refactoring that everywhere.

Just tested with DEFAULT_PORT 0 (ephemeral) and DEFAULT_PORT 4242 (fixed) and looks like we are good to go.

Ephemeral Port:

$ avahi-browse -r -t _zephyr._tcp
+ lowpan0 IPv6 zephyr                                        _zephyr._tcp         local
= lowpan0 IPv6 zephyr                                        _zephyr._tcp         local
   hostname = [zephyr.local]
   address = [2001:db8::1]
   port = [50271]
   txt = []

Fixed Port:

$ avahi-browse -r -t _zephyr._tcp
+ lowpan0 IPv6 zephyr                                        _zephyr._tcp         local
= lowpan0 IPv6 zephyr                                        _zephyr._tcp         local
   hostname = [zephyr.local]
   address = [2001:db8::1]
   port = [4242]
   txt = []

[cfriedt]

LGTM, pretty neat

Thanks - yeah, I think it's a relatively small change that should enable a lot of functionality :-) Especially judging by how some of the existing smart home devices use DNS-SD, I think it could really enable a lot of products to come out of Zephyr.