Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store: Add mixin to enable caching bucket #122

Merged
merged 9 commits into from
May 20, 2020
Merged

store: Add mixin to enable caching bucket #122

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
98766ee
Add store withCachingBucketMemcached mixin
kakkoyun May 15, 2020
25ff25a
Enable to specify different memcached backends for index cache and ca…
kakkoyun May 15, 2020
6838359
Bump version to minimum supported that includes added feature
kakkoyun May 15, 2020
99f13d8
Update Changelog
kakkoyun May 15, 2020
c554387
Bump version and reflect latest changes
kakkoyun May 18, 2020
ba175b3
Remove unused default value
kakkoyun May 18, 2020
a4d3946
Update configuration after recent changes
kakkoyun May 19, 2020
cb6f37b
Omit changes when memcached not specified
kakkoyun May 20, 2020
128e08a
Use Thanos v0.13.0-rc.0
kakkoyun May 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,17 @@ We use *breaking* word for marking changes that are not backward compatible (rel

- [#118](https://github.com/thanos-io/kube-thanos/pull/118) receive: Extend shutdown grace period to 900s (15min).

- [#122](https://github.com/thanos-io/kube-thanos/pull/122) store: Rename `withMemcachedIndexCache` to `withIndexCacheMemcached`

### Added

- [#105](https://github.com/thanos-io/kube-thanos/pull/105) compactor, store: Add deduplication replica label flags and delete delay labels

- [#105](https://github.com/thanos-io/kube-thanos/pull/105) compactor, store: Add deduplication replica label flags and delete delay labels

- [#119](https://github.com/thanos-io/kube-thanos/pull/119) receive: Distribute receive instances across node zones via pod anti affinity (note: only available on 1.17+).
- [#119](https://github.com/thanos-io/kube-thanos/pull/119) receive: Distribute receive instances across node zones via pod anti affinity (note: only available on Kubernetes 1.17+)

- [#122](https://github.com/thanos-io/kube-thanos/pull/122) store: Enable caching bucket support

### Fixed

Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ local commonConfig = {
config+:: {
local cfg = self,
namespace: 'thanos',
version: 'v0.12.0',
version: 'master-2020-05-18-6cfcb345',
image: 'quay.io/thanos/thanos:' + cfg.version,
objectStorageConfig: {
name: 'thanos-objectstorage',
Expand Down
6 changes: 4 additions & 2 deletions all.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ local commonConfig = {
config+:: {
local cfg = self,
namespace: 'thanos',
version: 'v0.12.0',
version: 'master-2020-05-18-6cfcb345',
image: 'quay.io/thanos/thanos:' + cfg.version,
objectStorageConfig: {
name: 'thanos-objectstorage',
Expand Down Expand Up @@ -83,7 +83,9 @@ local s =
local swm =
t.store +
t.store.withVolumeClaimTemplate +
t.store.withServiceMonitor + t.store.withMemcachedIndexCache +
t.store.withServiceMonitor +
t.store.withIndexCacheMemcached +
t.store.withCachingBucketMemcached +
commonConfig + {
config+:: {
name: 'thanos-store',
Expand Down
2 changes: 1 addition & 1 deletion example.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ local commonConfig = {
config+:: {
local cfg = self,
namespace: 'thanos',
version: 'v0.12.0',
version: 'master-2020-05-18-6cfcb345',
image: 'quay.io/thanos/thanos:' + cfg.version,
objectStorageConfig: {
name: 'thanos-objectstorage',
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-bucket-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-bucket-debugging
app.kubernetes.io/instance: thanos-bucket
app.kubernetes.io/name: thanos-bucket
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-bucket
namespace: thanos
spec:
Expand All @@ -21,7 +21,7 @@ spec:
app.kubernetes.io/component: object-store-bucket-debugging
app.kubernetes.io/instance: thanos-bucket
app.kubernetes.io/name: thanos-bucket
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
containers:
- args:
Expand All @@ -34,7 +34,7 @@ spec:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 4
httpGet:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-bucket-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-bucket-debugging
app.kubernetes.io/instance: thanos-bucket
app.kubernetes.io/name: thanos-bucket
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-bucket
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-compact-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-compactor
app.kubernetes.io/instance: thanos-compact
app.kubernetes.io/name: thanos-compact
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-compact
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-compact-serviceMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-compactor
app.kubernetes.io/instance: thanos-compact
app.kubernetes.io/name: thanos-compact
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-compact
namespace: thanos
spec:
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-compact-statefulSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-compactor
app.kubernetes.io/instance: thanos-compact
app.kubernetes.io/name: thanos-compact
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-compact
namespace: thanos
spec:
Expand All @@ -22,7 +22,7 @@ spec:
app.kubernetes.io/component: database-compactor
app.kubernetes.io/instance: thanos-compact
app.kubernetes.io/name: thanos-compact
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
containers:
- args:
Expand All @@ -37,7 +37,7 @@ spec:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 4
httpGet:
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-query-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: query-layer
app.kubernetes.io/instance: thanos-query
app.kubernetes.io/name: thanos-query
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-query
namespace: thanos
spec:
Expand All @@ -21,7 +21,7 @@ spec:
app.kubernetes.io/component: query-layer
app.kubernetes.io/instance: thanos-query
app.kubernetes.io/name: thanos-query
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
affinity:
podAntiAffinity:
Expand All @@ -47,7 +47,7 @@ spec:
- --store=dnssrv+_grpc._tcp.thanos-receive.thanos.svc.cluster.local
- --store=dnssrv+_grpc._tcp.thanos-rule.thanos.svc.cluster.local
- --store=dnssrv+_grpc._tcp.thanos-store.thanos.svc.cluster.local
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 4
httpGet:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-query-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: query-layer
app.kubernetes.io/instance: thanos-query
app.kubernetes.io/name: thanos-query
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-query
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-query-serviceMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: query-layer
app.kubernetes.io/instance: thanos-query
app.kubernetes.io/name: thanos-query
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-query
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-receive-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-write-hashring
app.kubernetes.io/instance: thanos-receive
app.kubernetes.io/name: thanos-receive
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-receive
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-receive-serviceMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-write-hashring
app.kubernetes.io/instance: thanos-receive
app.kubernetes.io/name: thanos-receive
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-receive
namespace: thanos
spec:
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-receive-statefulSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: database-write-hashring
app.kubernetes.io/instance: thanos-receive
app.kubernetes.io/name: thanos-receive
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-receive
namespace: thanos
spec:
Expand All @@ -22,7 +22,7 @@ spec:
app.kubernetes.io/component: database-write-hashring
app.kubernetes.io/instance: thanos-receive
app.kubernetes.io/name: thanos-receive
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
affinity:
podAntiAffinity:
Expand Down Expand Up @@ -75,7 +75,7 @@ spec:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 8
httpGet:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-rule-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: rule-evaluation-engine
app.kubernetes.io/instance: thanos-rule
app.kubernetes.io/name: thanos-rule
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-rule
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-rule-serviceMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: rule-evaluation-engine
app.kubernetes.io/instance: thanos-rule
app.kubernetes.io/name: thanos-rule
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-rule
namespace: thanos
spec:
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-rule-statefulSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: rule-evaluation-engine
app.kubernetes.io/instance: thanos-rule
app.kubernetes.io/name: thanos-rule
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-rule
namespace: thanos
spec:
Expand All @@ -22,7 +22,7 @@ spec:
app.kubernetes.io/component: rule-evaluation-engine
app.kubernetes.io/instance: thanos-rule
app.kubernetes.io/name: thanos-rule
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
containers:
- args:
Expand All @@ -44,7 +44,7 @@ spec:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 24
httpGet:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-store-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-store
namespace: thanos
spec:
Expand Down
2 changes: 1 addition & 1 deletion examples/all/manifests/thanos-store-serviceMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-store
namespace: thanos
spec:
Expand Down
28 changes: 25 additions & 3 deletions examples/all/manifests/thanos-store-statefulSet-with-memcached.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-store
namespace: thanos
spec:
Expand All @@ -22,7 +22,7 @@ spec:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
containers:
- args:
Expand All @@ -45,13 +45,35 @@ spec:
"max_item_size": "1MiB"
"timeout": "500ms"
"type": "MEMCACHED"
- |-
--store.caching-bucket.config="backend": "memcached"
"backend_config":
"addresses":
- "dnssrv+_client._tcp.<MEMCACHED_SERCIVE>.thanos.svc.cluster.local"
"dns_provider_update_interval": "10s"
"max_async_buffer_size": 10000
"max_async_concurrency": 20
"max_get_multi_batch_size": 0
"max_get_multi_concurrency": 100
"max_idle_connections": 100
"max_item_size": "1MiB"
"timeout": "500ms"
"blocks_iter_ttl": "5m"
"chunk_object_size_ttl": "24h"
"chunk_subrange_size": 16000
"chunk_subrange_ttl": "24h"
"max_chunks_get_range_requests": 3
"metafile_content_ttl": "24h"
"metafile_doesnt_exist_ttl": "15m"
"metafile_exists_ttl": "2h"
"metafile_max_size": "1MiB"
env:
- name: OBJSTORE_CONFIG
valueFrom:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 8
httpGet:
Expand Down
6 changes: 3 additions & 3 deletions examples/all/manifests/thanos-store-statefulSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
name: thanos-store
namespace: thanos
spec:
Expand All @@ -22,7 +22,7 @@ spec:
app.kubernetes.io/component: object-store-gateway
app.kubernetes.io/instance: thanos-store
app.kubernetes.io/name: thanos-store
app.kubernetes.io/version: v0.12.0
app.kubernetes.io/version: master-2020-05-18-6cfcb345
spec:
containers:
- args:
Expand All @@ -37,7 +37,7 @@ spec:
secretKeyRef:
key: thanos.yaml
name: thanos-objectstorage
image: quay.io/thanos/thanos:v0.12.0
image: quay.io/thanos/thanos:master-2020-05-18-6cfcb345
livenessProbe:
failureThreshold: 8
httpGet:
Expand Down
2 changes: 1 addition & 1 deletion examples/thanos-receive.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ t.receive {
local tr = self,
name:: 'thanos-receive',
namespace:: 'observability',
version:: 'v0.12.0-rc.0',
version:: 'master-2020-05-18-6cfcb345',
image:: 'quay.io/thanos/thanos:v' + tr.version,
replicas:: 3,
replicationFactor:: 3,
Expand Down
Loading