Validation error when starting Cortex 1.18.1 - max_series_per_query

[EpiJunkie]

When upgrading to 1.18.1 I noticed a validation error preventing our ingesters from starting.

Error:

ingester ts=2024-10-18T17:00:21.383332361Z caller=main.go:199 level=info msg="Starting Cortex" version="(version=1.18.1, branch=HEAD, revision=3f03510)"
ingester ts=2024-10-18T17:00:21.383663957Z caller=server.go:319 level=info http=:80 grpc=:9095 msg="server listening on addresses"
ingester ts=2024-10-18T17:00:21.385432336Z caller=memberlist_client.go:399 level=info msg="Using memberlist cluster node name" name=ingester-2-33590dab
ingester ts=2024-10-18T17:00:21.385443304Z caller=ingester.go:727 level=info msg="TSDB idle compaction timeout set" timeout=1h2m52.457346898s
ingester ts=2024-10-18T17:00:21.385671205Z caller=module_service.go:64 level=info msg=initialising module=server
ingester ts=2024-10-18T17:00:21.38580831Z caller=module_service.go:64 level=info msg=initialising module=memberlist-kv
ingester ts=2024-10-18T17:00:21.385822385Z caller=module_service.go:64 level=info msg=initialising module=runtime-config
ingester ts=2024-10-18T17:00:21.3866273Z caller=cortex.go:447 level=error msg="module failed" module=runtime-config err="invalid service state: Failed, expected: Running, failure: failed to load runtime config: load file: yaml: unmarshal errors:\n  line 12: field max_series_per_query not found in type validation.plain\n  line 23: field max_series_per_query not found in type validation.plain\n  line 34: field max_series_per_query not found in type validation.plain\n  line 45: field max_series_per_query not found in type validation.plain\n  line 56: field max_series_per_query not found in type validation.plain\n  line 67: field max_series_per_query not found in type validation.plain\n  line 78: field max_series_per_query not found in type validation.plain\n  line 89: field max_series_per_query not found in type validation.plain\n  line 100: field max_series_per_query not found in type validation.plain\n  line 111: field max_series_per_query not found in type validation.plain\n  line 122: field max_series_per_query not found in type validation.plain"
ingester ts=2024-10-18T17:00:21.386652367Z caller=cortex.go:447 level=error msg="module failed" module=ingester-service err="failed to start ingester-service, because it depends on module runtime-config, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: failed to load runtime config: load file: yaml: unmarshal errors:\n  line 12: field max_series_per_query not found in type validation.plain\n  line 23: field max_series_per_query not found in type validation.plain\n  line 34: field max_series_per_query not found in type validation.plain\n  line 45: field max_series_per_query not found in type validation.plain\n  line 56: field max_series_per_query not found in type validation.plain\n  line 67: field max_series_per_query not found in type validation.plain\n  line 78: field max_series_per_query not found in type validation.plain\n  line 89: field max_series_per_query not found in type validation.plain\n  line 100: field max_series_per_query not found in type validation.plain\n  line 111: field max_series_per_query not found in type validation.plain\n  line 122: field max_series_per_query not found in type validation.plain"
ingester ts=2024-10-18T17:00:21.51801446Z caller=memberlist_client.go:575 level=info msg="joined memberlist cluster" reached_nodes=16
ingester ts=2024-10-18T17:00:21.518061901Z caller=memberlist_client.go:612 level=info msg="leaving memberlist cluster"
ingester ts=2024-10-18T17:00:22.587875058Z caller=module_service.go:96 level=info msg="module stopped" module=memberlist-kv
ingester ts=2024-10-18T17:00:22.588050369Z caller=server_service.go:50 level=info msg="server stopped"
ingester ts=2024-10-18T17:00:22.588070446Z caller=module_service.go:96 level=info msg="module stopped" module=server
ingester ts=2024-10-18T17:00:22.588085023Z caller=cortex.go:436 level=info msg="Cortex stopped"
ingester ts=2024-10-18T17:00:22.588144426Z caller=log.go:121 level=error msg="error running cortex" err="failed services\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/__w/cortex/cortex/pkg/cortex/cortex.go:482\nmain.main\n\t/__w/cortex/cortex/cmd/cortex/main.go:201\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1695"
Stream closed EOF for default-ns/ingester-2 (ingester)

The solution seems to be removing max_series_per_query from the overrides.yaml. This is defined in various places in cortex/config.libsonnet.

This seems to have been removed as a configuration option in: cortexproject/cortex#5940