DNS lookups should not be case sensitive.

[tomwilkie]

Fixes #1461

[tomwilkie]

@inercia would you mind giving this a look please?

[rade]

What about the domain? Should that be treated case insensitively too?

Also, what's the motivation for dealing with the case insensitivity in the various lookups/comparisons, instead of simply lower-casing the hostname on entry creation?

[tomwilkie]

What about the domain? Should that be treated case insensitively too?

Domain is included in the hostname here (see https://github.com/weaveworks/weave/blob/master/weave#L844)

Also, what's the motivation for dealing with the case insensitivity in the various lookups/comparisons, instead of simply lower-casing the hostname on entry creation?

Backwards compatibility.

[tomwilkie]

Also, what's the motivation for dealing with the case insensitivity in the various lookups/comparisons, instead of simply lower-casing the hostname on entry creation?

Actually this will change the sort order, which means it isn't backwards compatible. Will change to lowercasing on entry creation.

[rade]

Will change to lowercasing on entry creation.

So where does that leave us w.r.t. backward compatibility?

[tomwilkie]

So where does that leave us w.r.t. backward compatibility?

If you are running a mixed 1.1/1.2 cluster

• the 1.1 host will resolve lower case versions of the containers on the weave 1.2 hosts
• if you are using non-lowercase hostnames/domain on a weave 1.1 hosts, the 1.2 host will not be able to resolve those names.

(1) is probably fine, but (2) looks like a show stopper. We could solve it by:

• also looking for non-lowercased names in the lookup method (as well as lowercased names)
• going back to original scheme (don't lowercase on the way in, just on search), but keeping the case on sort. This would also fix (1).

[rade]

I reckon add/remove should be case-sensitive, but lookup shouldn't be, i.e. it should return all records that match case-insensitively. I suspect this is how normal DNS servers behave, e.g. you can give them a zone file with mixed case records and they will return case-insensitively matching ones on lookup. Worth checking though.

[tomwilkie]

I reckon add/remove should be case-sensitive

Agreed.

Attempt (3) - only do case insensitive matching on lookup. Re: backwards compatibility - sort order is preserved. From a 1.1 host, lookup will be case sensitive, and from a 1.2 host, it will not be.

[tomwilkie]

Actually this won't work either; sort on a string puts all the upper case before the lowercase:

http://play.golang.org/p/UXLptUu953

func main() {
    a := []string{"aaa", "Aaa", "bbb", "Bbb", "aAa", "aaA", "bBb", "bbB"}
    sort.Strings(a)
    fmt.Println(a)
}

[Aaa Bbb aAa aaA aaa bBb bbB bbb]

Back to drawing board.

[rade]

Actually this won't work either; sort on a string puts all the upper case before the lowercase

The objective is correct. Your implementation is broken.

[tomwilkie]

If we change the sort order the gossip messages will fail to merge with 1.1 hosts. So we can't change the sort order.

We could make lookup O(n), but that sucks.

[rade]

If we change the sort order the gossip messages will fail to merge with 1.1 hosts.

How would this manifest itself?

[tomwilkie]

Routers with different sort orders would drop the connection to each other (nameserver.go:218 would fail)

[rade]

Why not sort the received data instead of checking that it is sorted?

[bboreham]

If you're sorting case-sensitively, why would the order be different?

[rade]

If you're sorting case-sensitively, why would the order be different?

It wouldn't be but then you cannot do an efficient case _in_sensitive lookup.

[tomwilkie]

Why not sort the received data instead of checking that it is sorted?

We can do that in 1.2, but the doesn't help with 1.2 sending gossips to 1.1.

[rade]

Oh dear. Looks like we are in a bit of pickle here.

[dpw]

Why can't you decouple the data structure that you gossip (which needs to be case-sensitively sorted for compatibility) from the data structure that you do lookups in?

[tomwilkie]

@dpw we can, or course. But we haven't done that yet (as it is simpler, with less code, not to).

As part of the original design we discussed putting indexes on the entries data structure to accelerate non-hostname lookups (ie reverse lookups, or delete by container id). Perhaps, now that the order of the entires doesn't help for lookup, this is needed for lookups too?

[tomwilkie]

And a even simpler solution would be to do a case sensitive sort for gossips we send, and a case-insensitive sort on gossips we receive.

[dpw]

@dpw we can, or course. But we haven't done that yet (as it is simpler, with less code, not to).

Ok. I interpreted Matthias' "bit of a pickle" comment as suggesting that there was some genuine difficulty, not merely the need to write some code that you'd prefer not to. As you were then.

[tomwilkie]

@rade PTAL

Failures are in 602_proxy_docker_py_test and are unrelated.

[tomwilkie]

Have tested manually - ubuntu-14 is this branch, ubuntu-15 is 1.1:

vagrant@ubuntu-14:~/src/github.com/weaveworks/weave$ nslookup searchapp.weave.local 172.17.42.1
Server:     172.17.42.1
Address:    172.17.42.1#53

Name:   searchapp.weave.local
Address: 10.32.0.12
Name:   searchapp.weave.local
Address: 10.40.0.2
Name:   searchapp.weave.local
Address: 10.40.0.3
Name:   searchapp.weave.local
Address: 10.32.0.4
Name:   searchapp.weave.local
Address: 10.32.0.3

vagrant@ubuntu-14:~/src/github.com/weaveworks/weave$ weave status dns
app          10.40.0.7       63acc94a6eed 56:f7:a1:9a:90:ff
app          10.40.0.6       c4078e711876 56:f7:a1:9a:90:ff
app          10.32.0.7       09cf6c314865 ae:23:70:89:80:61
app          10.32.0.8       5a7ac5be5157 ae:23:70:89:80:61
client       10.40.0.10      9b34528aee2e 56:f7:a1:9a:90:ff
client       10.32.0.11      2073da265f48 ae:23:70:89:80:61
elasticsearch 10.32.0.2       661f8e294fef ae:23:70:89:80:61
frontend     10.40.0.8       23d34d5994e1 56:f7:a1:9a:90:ff
frontend     10.40.0.9       52763050e15c 56:f7:a1:9a:90:ff
frontend     10.32.0.10      d34742dd2a09 ae:23:70:89:80:61
frontend     10.32.0.9       ff88aad13e05 ae:23:70:89:80:61
qotd         10.40.0.5       4c0277bfbc51 56:f7:a1:9a:90:ff
qotd         10.32.0.6       0dee827d5b34 ae:23:70:89:80:61
redis        10.40.0.4       14c131aec9e3 56:f7:a1:9a:90:ff
redis        10.32.0.5       038b5dd8171a ae:23:70:89:80:61
searchapp    10.40.0.2       97992c807504 56:f7:a1:9a:90:ff
searchapp    10.40.0.3       fa539363ed99 56:f7:a1:9a:90:ff
searchapp    10.32.0.3       35ec6a8ea094 ae:23:70:89:80:61
Searchapp    10.32.0.12      4e90d538db02 ae:23:70:89:80:61
searchapp    10.32.0.4       940d037096b5 ae:23:70:89:80:61

vagrant@ubuntu-15:~/src/github.com/weaveworks/scope$ weave status dns
Searchapp    10.32.0.12      4e90d538db02 ae:23:70:89:80:61
app          10.40.0.7       63acc94a6eed 56:f7:a1:9a:90:ff
app          10.40.0.6       c4078e711876 56:f7:a1:9a:90:ff
app          10.32.0.7       09cf6c314865 ae:23:70:89:80:61
app          10.32.0.8       5a7ac5be5157 ae:23:70:89:80:61
client       10.40.0.10      9b34528aee2e 56:f7:a1:9a:90:ff
client       10.32.0.11      2073da265f48 ae:23:70:89:80:61
elasticsearch 10.32.0.2       661f8e294fef ae:23:70:89:80:61
frontend     10.40.0.8       23d34d5994e1 56:f7:a1:9a:90:ff
frontend     10.40.0.9       52763050e15c 56:f7:a1:9a:90:ff
frontend     10.32.0.10      d34742dd2a09 ae:23:70:89:80:61
frontend     10.32.0.9       ff88aad13e05 ae:23:70:89:80:61
qotd         10.40.0.5       4c0277bfbc51 56:f7:a1:9a:90:ff
qotd         10.32.0.6       0dee827d5b34 ae:23:70:89:80:61
redis        10.40.0.4       14c131aec9e3 56:f7:a1:9a:90:ff
redis        10.32.0.5       038b5dd8171a ae:23:70:89:80:61
searchapp    10.40.0.2       97992c807504 56:f7:a1:9a:90:ff
searchapp    10.40.0.3       fa539363ed99 56:f7:a1:9a:90:ff
searchapp    10.32.0.3       35ec6a8ea094 ae:23:70:89:80:61
searchapp    10.32.0.4       940d037096b5 ae:23:70:89:80:61

[tomwilkie]

Failure in test is the docker py test_792_explicit_port_protocol failure, unrelated.