vm_memory_high_watermark is less than expected

[bdaoudtdc]

Describe the bug

I setup me resources as follows
resources:
requests:
cpu: 1
memory: 6Gi
limits:
cpu: 4
memory: 6Gi

My vm_memory_high_watermark is set to 0.7

I expected the watermark to be 0.7*6Gi , but what I see in the logs is

Memory high watermark set to 3440 MiB (3607772529 bytes) of 4915 MiB (5153960756 bytes) total

Not sure why the max is set to 4915 MiB and not to 6144 MiB

When I login to the container , I see the full amount that is assigned to the host
rabbitmq@rmq-qa-cluster-server-2:/etc$ free
total used free shared buff/cache available
Mem: 16247656 8008900 2975708 869608 5263048 7031112
Swap: 0 0 0

Version and environment information

• RabbitMQ: [3.8.12]
• RabbitMQ Cluster Operator: [1.5.0]
• Kubernetes: [1.17.13]

[ChunyiLyu]

Hi @bdaoudtdc, I have converted the issue to a discussion since this is expected behavior and not a bug. A while back we added a feature to cluster operator so it provides headroom for the memory limit (see:

cluster-operator/internal/resource/configmap_test.go

Lines 505 to 516 in c813d56

	Context("Memory Limits", func() {
	It("sets a RabbitMQ memory limit with headroom when memory limits are specified", func() {
	const GiB int64 = 1073741824
	instance.ObjectMeta.Name = "rabbit-mem-limit"
	instance.Spec.Resources.Limits = map[corev1.ResourceName]k8sresource.Quantity{corev1.ResourceMemory: k8sresource.MustParse("10Gi")}
	expectedConfiguration := iniString(fmt.Sprintf("total_memory_available_override_value = %d", 8*GiB))
	Expect(configMapBuilder.Update(configMap)).To(Succeed())
	Expect(configMap.Data).To(HaveKeyWithValue("userDefinedConfiguration.conf", expectedConfiguration))
	})
	})

). This is to avoid having pods killed by k8s with OOM kill because RabbitMQ may temporary exceeds its high memory water mark. The headroom we've decided on is 20% with a maximum of 2GB and that's why the memory high watermark and total memory is different from what you've expected.