Skip to content

Commit

Permalink
[Docs] Druid 29.0.0 release notes (apache#15805)
Browse files Browse the repository at this point in the history
Co-authored-by: 317brian <[email protected]>
  • Loading branch information
ektravel and 317brian authored Feb 21, 2024
1 parent 4e7493e commit b5577df
Show file tree
Hide file tree
Showing 6 changed files with 672 additions and 14 deletions.
Binary file added docs/release-info/assets/image01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/release-info/assets/image02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/release-info/assets/image03.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
602 changes: 588 additions & 14 deletions docs/release-info/release-notes.md

Large diffs are not rendered by default.

63 changes: 63 additions & 0 deletions docs/release-info/upgrade-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,69 @@ The upgrade notes assume that you are upgrading from the Druid version that imme

For the full release notes for a specific version, see the [releases page](https://github.com/apache/druid/releases).

## 29.0.0

### Upgrade notes

#### Changed `equals` filter for native queries

The [equality filter](https://druid.apache.org/docs/latest/querying/filters#equality-filter) on mixed type `auto` columns that contain arrays must now be filtered as their presenting type. This means that if any rows are arrays (for example, the segment metadata and `information_schema` reports the type as some array type), then the native queries must also filter as if they are some array type.

This change impacts mixed type `auto` columns that contain both scalars and arrays. It doesn't impact SQL, which already has this limitation due to how the type presents itself.

[#15503](https://github.com/apache/druid/pull/15503)

#### Console automatically sets `arrayIngestMode` for MSQ queries

Druid console now configures the `arrayIngestMode` parameter in the data loading flow, and its value can persist across the SQL tab unless manually updated. When loading multi-value dimensions or arrays in the Druid console, note the value of the `arrayIngestMode` parameter to prevent mixing multi-value dimensions and arrays in the same column of a data source.

[#15588](https://github.com/apache/druid/pull/15588)

#### Improved concurrent append and replace (experimental)

You no longer have to manually determine the task lock type for concurrent append and replace (experimental) with the `taskLockType` task context. Instead, Druid can now determine it automatically for you. You can use the context parameter `"useConcurrentLocks": true` for individual tasks and datasources or enable concurrent append and replace at a cluster level using `druid.indexer.task.default.context`.

[#15684](https://github.com/apache/druid/pull/15684)

#### Enabled empty ingest queries

The MSQ task engine now allows empty ingest queries by default. For queries that don't generate any output rows, the MSQ task engine reports zero values for `numTotalRows` and `totalSizeInBytes` instead of null. Previously, ingest queries that produced no data would fail with the `InsertCannotBeEmpty` MSQ fault.

To revert to the original behavior, set the MSQ query parameter `failOnEmptyInsert` to `true`.

[#15495](https://github.com/apache/druid/pull/15495) [#15674](https://github.com/apache/druid/pull/15674)

#### Enabled query request queuing by default when total laning is turned on

When query scheduler threads are less than server HTTP threads, total laning turns on.
This reserves some HTTP threads for non-query requests such as health checks.
The total laning previously would reject any query request that exceeds the lane capacity.
Now, excess requests will instead be queued with a timeout equal to `MIN(Integer.MAX_VALUE, druid.server.http.maxQueryTimeout)`.

[#15440](https://github.com/apache/druid/pull/15440)

#### Changed how empty or null array columns are stored

Columns ingested with the auto column indexer that contain only empty or null arrays are now stored as `ARRAY<LONG\>` instead of `COMPLEX<json\>`.

[#15505](https://github.com/apache/druid/pull/15505)

#### Changed how Druid allocates weekly segments

When the requested granularity is a month or larger but a segment can't be allocated, Druid resorts to day partitioning.
Unless explicitly specified, Druid skips week-granularity segments for data partitioning because these segments don't align with the end of the month or more coarse-grained intervals.

Previously, if Druid couldn't allocate segments by month, it tried allocating them by week next.
In the new behavior, Druid skips partitioning by week and goes directly to day. Week segments can only be allocated if the chosen partitioning in the append task is WEEK.

[#15589](https://github.com/apache/druid/pull/15589)

### Removed the `auto` search strategy

Removed the `auto` search strategy from the native search query. Setting `searchStrategy` to `auto` is now equivalent to `useIndexes`.

[#15550](https://github.com/apache/druid/pull/15550)

## 28.0.0

### Upgrade notes
Expand Down
21 changes: 21 additions & 0 deletions website/.spelling
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ CloudWatch
ColumnDescriptor
Corretto
CLI
CVE
CVEs
DDL
DML
DNS
Expand Down Expand Up @@ -104,6 +106,7 @@ GPG
GSSAPI
GUIs
GroupBy
Gauva
Guice
HDFS
HDFSFirehose
Expand Down Expand Up @@ -141,6 +144,7 @@ JDBC
JDK
JDK7
JDK8
Jedis
JKS
jks
JMX
Expand Down Expand Up @@ -185,6 +189,7 @@ Murmur3
MVCC
MV_TO_ARRAY
NFS
NPE
OCF
OIDC
OLAP
Expand Down Expand Up @@ -580,6 +585,7 @@ versioning
virtualColumns
w.r.t.
walkthrough
webpack
whitelist
whitelisted
whitespace
Expand Down Expand Up @@ -2174,6 +2180,21 @@ UserGroupInformation
CVE-2019-17571
CVE-2019-12399
CVE-2018-17196
GHSA-mm8h-8587-p46h
GHSA-q5
GHSA-f62v-xpxf-3v68
GHS
GHSA-4p6w-m9wc-c9c9
GHSA-q5r4-cfpx-h6fh
GHSA-5v34-g2px-j4fw
GHSA-cgwf-w82q-5jrr
GHSA-7g24-qg88-p43q
GHSA-jgvc-jfgh-rjvv
GHSA-cqj8-47ch-rvvq
CVE-2022-24329
CVE-2022-42003
CVE-2022-42004
CVE-2022-25647
bin.tar.gz
0s
1T
Expand Down

0 comments on commit b5577df

Please sign in to comment.