v0.18.1

Changelog

New Features

• 575e11c: feat: use WithOpenTelemetryZap (@ernado)

Docker images

ghcr.io/go-faster/oteldb:v0.18.1

v0.18.0

Changelog

Docker images

ghcr.io/go-faster/oteldb:v0.18.0

v0.17.0

Changelog

New Features

• 35afe5c: feat(chdump): add Clickhouse dump reader (@tdakkota)
• 8f5dbb3: feat(chtracker): retrive reports concurrently (@tdakkota)
• 1acf68d: feat(logqlbench): add label queries (@tdakkota)

Bug fixes

• 22ac590: fix(chstorage): set TTL field (@ernado)
• 365247d: fix(otelbench): add query type to report (@tdakkota)
• d7ea501: fix(otelbench): note that query is failed due to timeout (@tdakkota)
• f93e8f8: fix(otelbench): properly handle lack of reports (@tdakkota)
• 69d0d53: fix: properly use iterators (@tdakkota)

CI/CD updates

• 81e9cba: ci(e2e): update Codecov action options (@tdakkota)

Other work

• 67796ba: refactor(chstorage): use iter.Seq, when possible (@tdakkota)
• d8baaeb: refactor(chtrace): use iter.Seq, when possible (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.17.0

v0.16.0

Changelog

New Features

• f341ca4: feat(ch-log-bench-read): setup a flow for dump benchmarking (@tdakkota)
• f5311cc: feat(chsql): add GROUP BY (@tdakkota)
• b7ffb8a: feat(chstorage): generate DDL (@ernado)
• 2713c95: feat(chstorage): non low-cardinality attrs (@ernado)
• 5fc2dd7: feat(chstorage): use ddl generation (@ernado)
• fdb5326: feat(otelbench): add dump management (@ernado)
• e81bbab: feat(otelbench): add restore (@ernado)
• 936bc51: feat(otelbench): implement dump limits (@ernado)
• 87973fe: feat(otelbench): use zstd compression (@ernado)
• c2ff524: feat(oteldb): update pgo (@ernado)

Bug fixes

• e39a67b: fix(chstorage): use unmapped label name for logs selection (@tdakkota)
• 2639e7a: fix(logqlbench): handle timeouts properly (@tdakkota)
• 4570c40: fix(otelbench): fix limits (@ernado)

Performance updates

• 8505a42: Revert "perf: add some unnecessary indexes" (@ernado)
• 2a8ea8d: perf(chstorage): query labels and label values from deduplicated resource (@tdakkota)
• bf573c9: perf(chstorage): query only resource for logs (@ernado)
• a630a6a: perf(chstorage): use GROUP BY hack to deduplicate span names (@tdakkota)
• 9b52dc8: perf(chstorage): use keys index for logs attributes (@tdakkota)
• 6fcd8b4: perf: add some unnecessary indexes (@ernado)

Documentation updates

• b805840: docs(arch): add note (@ernado)
• 4ae113b: docs(benchmarking): add data retrieval (@ernado)

Benchmarking tooling changes

• 9b0f324: ci(bench): try running on self-hosted (@ernado)

CI/CD updates

• 91b1a71: Revert "ci: unknown flag remove orphans" (@ernado)
• 85cf0ce: ci(deploy): bump k8s version (@ernado)
• d932795: ci(k8s): disable pyroscope for now (@ernado)
• b3f3dd6: ci(k8s): enable pprof addr (@ernado)
• cd1468a: ci(k8s): enable pyroscrope (@ernado)
• 34cbac2: ci(k8s): reduce ttl to 2h (@ernado)
• 048543e: ci(otel): rebalance resources (@ernado)
• fc5e0ff: ci(otel): reduce replicas (@ernado)
• e10a00d: ci: bump (@ernado)
• c3a4404: ci: bump (@ernado)
• 7b84e1a: ci: unknown flag remove orphans (@ernado)

Docker images

ghcr.io/go-faster/oteldb:v0.16.0

v0.15.1

Changelog

Bug fixes

• b3c49ae: fix: disable memory_ballast (@ernado)

Benchmarking tooling changes

• e74c21d: ci(bench): check bench not broken (@ernado)

CI/CD updates

• c630a76: ci(deploy)): bump go (@ernado)
• 956d945: ci(deploy): not broken anymore (@ernado)
• 32f868d: ci(deploy): otelbot go (@ernado)
• 14f2054: ci(release): bump go (@ernado)

Docker images

ghcr.io/go-faster/oteldb:v0.15.1

v0.15.0

Changelog

New Features

• 941e22c: feat(autometric): add package for automatic metric initialization (@tdakkota)
• c804add: feat(chsql): add string search token iterator (@tdakkota)
• 998771c: feat(chstorage): collect tokens from line filter (@tdakkota)
• 368705c: feat(chstorage): instrument inserter using autometric (@tdakkota)
• 780cfdd: feat(chstorage): offload unaffected label filters (@tdakkota)
• c10f915: feat(cliversion): add package to retrive debug version info (@tdakkota)
• 954b9f7: feat(docker-logql): make docker-logql part of oteldb (@tdakkota)
• 84f765a: feat(dockerlog): add steam label to entries (@tdakkota)
• 74227df: feat(dockerlog): add package to query logs from Docker daemon (@tdakkota)
• 6c30131: feat(oas): add infos to prometheus responses (@tdakkota)
• a0e7de8: feat(otelstorage): add Len method to Attrs (@tdakkota)
• f5652b1: feat(typedpool): add typed sync.Pool wrapper (@tdakkota)

Bug fixes

• 7a36d6a: fix(chstorage): check buckets size for overflow (@tdakkota)
• 69e8223: fix(chstorage): do not use hasToken if line filter value is an ID (@tdakkota)
• 7325edd: fix(chstorage): properly initialize inserts metric (@tdakkota)
• afd266b: fix(chstorage): re-use query_id when retrying (@tdakkota)
• 185f1a0: fix(chstorage): update querier due to changes in Prometheus storage API (@tdakkota)
• 68c4ef1: fix(chstorage): use right enum values (@tdakkota)
• 86424df: fix(logql): properly count regexp captures (@tdakkota)
• 75199df: fix(logqlpattern): properly count named captures (@tdakkota)
• cafb86e: fix(otelbench): check packet for overflow (@tdakkota)
• a806327: fix(oteldb): bind OTLP receivers on all interfaces by default (@tdakkota)
• 23a1341: fix(oteldb): remove UseLocalHostAsDefaultHost feature gate (@tdakkota)
• 1364027: fix(otelstorage): replace unicode characters with underscore (@tdakkota)
• db7462d: fix(promhandler): support limit parameter (@tdakkota)

Performance updates

• 753e84d: perf(chstorage): compute column list only once (@tdakkota)
• 154ac12: perf(chstorage): generate insert body only once (@tdakkota)
• 6eecac1: perf(chstorage): re-use log inserter columns (@tdakkota)
• b635ad8: perf(chstorage): re-use log inserter columns proto.Input (@tdakkota)
• a249e32: perf(otelstorage): increase Attrs on-stack sorting threshold (@tdakkota)
• db78d4b: perf(otelstorage): optimize replacer (@tdakkota)
• 34b4988: perf(otelstorage): write converted label right to column (@tdakkota)

CI/CD updates

• 364829e: ci(golangci-lint): disable G115 check (@tdakkota)
• 72eb42b: ci: run bench on github runners (@ernado)

Updates to tests

• 253c790: test(dockerlog): add ParseLog benchmark (@tdakkota)
• b65379e: test(lokie2e): add inserter benchmark (@tdakkota)
• 9eceb31: test(lokie2e): ensure that tokens are collected (@tdakkota)
• 00f4c17: test(otelstorage): add KeyToLabel benchmark (@tdakkota)
• c311fe4: test(otelstorage): ensure that Unicode is properly handled (@tdakkota)
• 5ff9560: test(tempoe2e): add traces insereter benchmark (@tdakkota)
• 06783f7: test(tempoe2e): compare span attributes (@tdakkota)

Other work

• 102fd87: refactor(logstorage): create batch writer (@tdakkota)
• a18c67c: refactor(tracestorage): create batch writer (@tdakkota)
• 41a1d54: refactor(xsync): move all sync utilities to xsync (@tdakkota)
• 8da36bf: refactor: use xsync.Pool instead of sync.Pool (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.15.0

v0.14.1

Changelog

Bug fixes

• 0cc18ab: fix(chstorage): pass valid exemplar scope attributes (@tdakkota)
• b13277c: fix(otelstorage): properly handle zero value map hashing (@tdakkota)

Updates to tests

• 59966df: test(prome2e): insert exemplar to ensure inserter works properly (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.14.1

v0.14.0

Changelog

New Features

• 4885d72: feat(chsql): add CAST function (@tdakkota)
• 4753363: feat(chstorage): offload OR line filters (@tdakkota)
• 1bdd70f: feat(chstorage): offload label predicates (@tdakkota)
• d3c7238: feat(lexer): add @explain token (@tdakkota)
• b255f2b: feat(logql): add explain expression (@tdakkota)
• c25de28: feat(logql): implement fmt.Stringer for LabelPredicate (@tdakkota)
• 1a67c55: feat(logqlengine): explain query (@tdakkota)
• f59f962: feat(logqlengine): implement or in line filters (@tdakkota)
• a72d0fd: feat(logqlengine): use FastRegexMatcher for label matching (@tdakkota)

Bug fixes

• d9bf1d7: fix(chstorage): cast String to LowCardinality(String) (@tdakkota)
• 7b9af32: fix(chstorage): do not offload label predicates after line format (@tdakkota)
• 3266f22: fix(chstorage): use proper column type when querying samples (@tdakkota)
• 5b7ffa7: fix(logql): incorrect String for BinOp (@tdakkota)
• c06e3c9: fix(logqlengine): handle mapped IPv4 address (@tdakkota)
• 721e905: fix(logqlengine): properly handle negated OR line filter (@tdakkota)
• 94ace20: fix(logqlengine): properly negate IP matcher (@tdakkota)
• cf06b0a: fix(oteldb): disalbe UseLocalHostAsDefaultHost feature gate (@tdakkota)

CI/CD updates

• dade24f: ci(compliance): rebuild image if needed (@tdakkota)

Updates to tests

• ca2673b: test(ch-logql-compliance): add OR lip line filter test (@tdakkota)
• a04597b: test(ch-logql-compliance): add negated line filter test (@tdakkota)
• fd980f5: test(logqlengine): add more line filter tests (@tdakkota)
• 16a1c77: test(logqlengine): update test due to API changes (@tdakkota)
• 9049a51: test(lokie2e): add OR line filter test (@tdakkota)
• 0a01405: test(lokie2e): add trace_id/span_id lookup tests (@tdakkota)
• e40c2d9: test(lokie2e): add explain query test (@tdakkota)
• 213b469: test(lokie2e): add more label predicate tests (@tdakkota)
• 0db5f62: test(lokie2e): ensure empty sample results are well-handled (@tdakkota)

Other work

• 9f61d6f: refactor(logql): compile pattern during parsing (@tdakkota)
• 30d395b: refactor(logqlengine): use UnparenLabelPredicate (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.14.0

v0.13.0

Changelog

New Features

• cba958c: feat(ch-bench-read): add Series benchmark (@tdakkota)
• c5422c3: feat(ch-bench-read): add option to set benchmark runs (@tdakkota)
• a2414c4: feat(ch-bench-read): add script to run benchmark locally (@tdakkota)
• 3fcc6d7: feat(chsql): add IsSingleToken to check whether string is a token (@tdakkota)
• 24a5d8d: feat(chsql): add ORDER BY expression (@tdakkota)
• 3473bf0: feat(chsql): add PREWHERE (@tdakkota)
• cef5f8f: feat(chsql): add hasToken and like functions (@tdakkota)
• 703a3e5: feat(chsql): add more helpers to build predicates (@tdakkota)
• ab07bf6: feat(chsql): add query builder package (@tdakkota)
• f9f9048: feat(chsql): add subquery support (@tdakkota)
• 2428d85: feat(chsql): use simpleJSONHas to find attributes (@tdakkota)
• 40896e2: feat(chstorage): lookup attribute scope before querying data (@tdakkota)
• a78de4d: feat(chstorage): offload Series (@tdakkota)
• ca0d905: feat(chstorage): query only labels when possible (@tdakkota)
• 2140e95: feat(iterators): add Reset method to SliceIterator (@tdakkota)
• 96619b4: feat(logql): implement fmt.Stringer for Selector (@tdakkota)
• a8580d9: feat(logqlbench): add LogQL benchmark tool (@tdakkota)
• 6010beb: feat(otelbench): add logql analyze command (@tdakkota)
• ed30e30: feat(otelbench): add option to seed log generator (@tdakkota)
• 86e10ec: feat(otelbench): collect received bytes and rows too (@tdakkota)
• ac16a14: feat(otelbench): teach promql analyze to print reports in benchstat format (@tdakkota)
• 459d963: feat(promhandler): do Labels/LabelValues queries in parallel (@tdakkota)
• c9e455e: feat(promhandler): query series concurrently (@tdakkota)
• 4a37f01: feat(promhandler): use optimized API for Series queries (@tdakkota)
• 1c77c7c: feat(traceql): implement stringer for Autocomplete (@tdakkota)
• 8c3dad8: feat(xattribute): add Duration to pass duration attribute (@tdakkota)
• 567c14c: feat(xattribute): add unix timestamp attribute (@tdakkota)

Bug fixes

• 1794195: fix(ch-bench-read): remove unnecessary -O flag (@tdakkota)
• 19bcf51: fix(chsql): convert expression to DateTime64 before comparing (@tdakkota)
• 6604bc7: fix(chsql): convert timestamp to DateTime64 before comparing (@tdakkota)
• b11d586: fix(chsql): do not wrap where/prewhere expression (@tdakkota)
• 36c3572: fix(chsql): ensure time is not zero in InTimeRange (@tdakkota)
• 7b7ddd0: fix(chsql): pass proper columns to results (@tdakkota)
• 5eb101c: fix(chsql): properly handle column aliasing (@tdakkota)
• 6c319ad: fix(chsql): put binary expression into parenthesis (@tdakkota)
• 84f90e9: fix(chstorage): add DISTINCT to series query (@tdakkota)
• 5665fe3: fix(chstorage): check for MinTime/MaxTime when querying exemplars (@tdakkota)
• 94ff446: fix(chstorage): convert log abel value to string (@tdakkota)
• 73773f0: fix(chstorage): do not return __name__ label if there is no match (@tdakkota)
• a7e6f79: fix(chstorage): do not return empty label names in log series (@tdakkota)
• d77b43b: fix(chstorage): do not return high cardinality labels as part of series (@tdakkota)
• f75bd30: fix(chstorage): do not skip level log label when returning series (@tdakkota)
• 4ad0b3a: fix(chstorage): filter root span names, when rootName is queried (@tdakkota)
• 2184b0a: fix(chstorage): handle autocompelete query when getting span names (@tdakkota)
• 01b5d25: fix(chstorage): handle trace-to-logs line filter properly (@tdakkota)
• fa245c7: fix(chstorage): hash exemplar scope attributes too (@tdakkota)
• 0a08770: fix(chstorage): lookup by scope labels too (@tdakkota)
• a37910e: fix(chstorage): lookup by stringified attribute value (@tdakkota)
• 8c264c8: fix(chstorage): make trace_id/span_id case-insensitive (@tdakkota)
• 80c232c: fix(chstorage): make label limit configurable (@tdakkota)
• 8a0c014: fix(chstorage): properly handle Prometheus min/max time (@tdakkota)
• cdb9ae0: fix(chstorage): properly handle matchers in LabelNames (@tdakkota)
• a130d78: fix(chstorage): properly handle matchers in LabelValues (@tdakkota)
• a62d8ad: fix(chstorage): properly negate matchers (@tdakkota)
• cedd89d: fix(chstorage): query scope attributes (@tdakkota)
• 1550ef9: fix(chstorage): remove trailing comma in migration table DDL (@tdakkota)
• d214c03: fix(chstorage): remove unnecessary label conversion (@tdakkota)
• a976b34: fix(chstorage): respect query parameter when selecting log label values (@tdakkota)
• e3116c5: fix(chstorage): save metrics scope attributes (@tdakkota)
• 64e7127: fix(chstorage): sort attributes keys before encoding (@tdakkota)
• ae6e549: fix(chstorage): sort log labels and label values before returning (@tdakkota)
• 9f3598e: fix(chstorage): sort metric labels and label values before returning (@tdakkota)
• 7be284b: fix(chstorage): use chsql for getMetricsLabelMapping too (@tdakkota)
• d637972: fix(chstorage): use normalized metric name to lookup (@tdakkota)
• bedbddf: fix(chstorage): use right column name for service.namespace (@tdakkota)
• 878b7f6: fix(dev): pass a proper image name when building (@tdakkota)
• 306b12d: fix(lokihandler): parse query parameter (@tdakkota)
• 48fc0a5: fix(lokihandler): use engine's parsing options (@tdakkota)
• fdc0ba4: fix(otelbench): avoid data race when collecting PromQL queries (@tdakkota)
• d912283: fix(otelbench): disallow positional arguments to avoid confusion (@tdakkota)
• d4ba238: fix(otelbench): return proper benchstat units (@tdakkota)
• 3adaf57: fix(otelbench): use query title to output, if available (@tdakkota)
• 17b6361: fix(promhandler): sort labels and label values after merge (@tdakkota)
• e430f2c: fix(promhandler): sort series before merging (@tdakkota)
• 4971482: fix(tempohandler): sort traces before returning (@tdakkota)
• a1a5a49: fix(traceqlengine): sort traces before returning (@tdakkota)

Performance updates

• 7e7fe5a: perf(chstorage): use hasToken to skip blocks (@tdakkota)
• 2082cc7: perf(logqlabels): use stack-allocated slices for keys (@tdakkota)
• d760d30: perf(logqlengine): re-use key buffer (@tdakkota)
• b085670: perf(logqlengine): use index map to avoid result slice allocation (@tdakkota)

Documentation updates

• b50f638: docs(ch-bench-read): describe how to compare changes (@tdakkota)
• 10af804: docs(integration): describe how to get debug traces for tests (@tdakkota)

Benchmarking tooling changes

• a9cc7fe: ci(bench): disable CheckBenchAPI until it fixed (@tdakkota)
• 7122c98: ci(ch-log-bench-read): add LogQL benchmark setup (@tdakkota)

CI/CD updates

• 11e62f4: ci(deploy): disable deploy until it fixed (@tdakkota)
• 02721b1: ci: increase deploy timeouts (@ernado)

Updates to tests

• b7e9bd8: test(chstorage): ensure that attributes are simpleJSON-compatible (@tdakkota)
• 137e1fb: test(logql): do not fail test unconditionally (@tdakkota)
• 2ebd4a4: test(logqlables): add groupEntries benchmark and test (@tdakkota)
• 60fcf9a: test(logqlengine): reduce cardinality in groupEntries benchmark (@Tdakk...

v0.12.2

Changelog

New Features

• f9f3da3: feat(logql): implement fmt.Stringer for matchers (@tdakkota)
• 04e265c: feat(oas): handle buildinfo requests to Tempo API (@tdakkota)
• 56280c0: feat(traceql): implement fmt.Stringer for matchers (@tdakkota)
• 16c696d: feat(xattribute): add utility package for building attributes (@tdakkota)

Bug fixes

• d2a006a: fix(lokicompliance): ignore stats (@tdakkota)
• 9398bdc: fix(lokihandler): return proper error status code (@tdakkota)
• e3ef2ef: fix(tempohandler): implement buildinfo handler (@tdakkota)

CI/CD updates

• 1dac28e: ci(compliance): try to upload compliance test result even if pipeline fails (@tdakkota)

Other work

• 1e68263: refactor(traceql): rename AsFloat to ToFloat (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.12.2