-
Notifications
You must be signed in to change notification settings - Fork 31
Configure Cassandra nodes to lookup their IP address by their fully qualified domain name #334
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -57,6 +57,7 @@ func StatefulSetForCluster( | |
Type: apps.RollingUpdateStatefulSetStrategyType, | ||
}, | ||
PodManagementPolicy: apps.OrderedReadyPodManagement, | ||
ServiceName: statefulSetName, | ||
Template: apiv1.PodTemplateSpec{ | ||
ObjectMeta: metav1.ObjectMeta{ | ||
Labels: nodePoolLabels, | ||
|
@@ -195,6 +196,34 @@ func StatefulSetForCluster( | |
Name: "HEAP_NEWSIZE", | ||
Value: "100M", | ||
}, | ||
// Deliberately set to a single space to force Cassandra to do a host name lookup. | ||
// See https://github.com/apache/cassandra/blob/cassandra-3.11.2/conf/cassandra.yaml#L592 | ||
{ | ||
Name: "CASSANDRA_LISTEN_ADDRESS", | ||
Value: " ", | ||
}, | ||
{ | ||
Name: "CASSANDRA_BROADCAST_ADDRESS", | ||
Value: " ", | ||
}, | ||
{ | ||
Name: "CASSANDRA_RPC_ADDRESS", | ||
Value: " ", | ||
}, | ||
// Set a non-existent default seed. | ||
// The Kubernetes Seed Provider will fall back to a default seed host if it can't look up seeds via the CASSANDRA_SERVICE. | ||
// And if the CASSANDRA_SEEDS environment variable is not set, it defaults to localhost. | ||
// Which could cause confusion if a non-seed node is temporarily unable to lookup the seed nodes from the service. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What would happen if we were to set this to localhost instead? Would the node fail to boot, or would it cause a split brain of some sort? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it would mean that new nodes might consider themselves seeds in the event that the KubernetesSeedProvider plugin encounters an error. |
||
// We want the list of seeds to be strictly controlled by the service. | ||
// See: | ||
// https://github.com/docker-library/cassandra/blame/master/3.11/docker-entrypoint.sh#L31 and | ||
// https://github.com/apache/cassandra/blob/cassandra-3.11.2/conf/cassandra.yaml#L416 and | ||
// https://github.com/kubernetes/examples/blob/cabf8b8e4739e576837111e156763d19a64a3591/cassandra/java/src/main/java/io/k8s/cassandra/KubernetesSeedProvider.java#L69 and | ||
// https://github.com/kubernetes/examples/blob/cabf8b8e4739e576837111e156763d19a64a3591/cassandra/go/main.go#L51 | ||
{ | ||
Name: "CASSANDRA_SEEDS", | ||
Value: "black-hole-dns-name", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm slightly concerned this could become an attack vector. What if someone creates a service named black-hole-dns-name? Could an attacker gain access to the contents of the Cassandra DB as a result? If possible, I'd rather set this to a value that causes Cassandra to fail hard/crash (as this value should only be referenced in the event of a DNS resolution failure) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the answer is to remove the fallback code from the Kubernetes Seed Provider. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll do this in a follow up branch, where I remove the reliance on the Docker image entrypoint and instead create a Navigator default config. |
||
}, | ||
{ | ||
Name: "CASSANDRA_ENDPOINT_SNITCH", | ||
Value: cassSnitch, | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we use the Cassandra RPC interface, and do we want to listen on the default pod IP for RPC connections? It seems to me like we'd only want to listen on 127.0.0.1?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't use it, but I'm not sure whether Cassandra nodes talk to each other with this protocol.
The default is to listen on all addresses:
So setting it here may not be ideal, but I can improve or remove it later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍