If a JSON document contains strings that can't fit in a single chunk, use the naive Blob chunker instead of the smart JSON chunker.#8723

Merged

nicktobey merged 15 commits intomainfrom nicktobey/json-big

Jan 9, 2025

+249-62

Commits on Jan 9, 2025

Add tests for inserting and reading large string values in JSON documents.
nicktobey
committed
In JSON chunker, allow chunking in the middle of large strings, and add a new location type representing the inside of a string.
nicktobey
committed
Detect when a json key is too large to chunk and don't use smart JSON chunking.
nicktobey
committed
Update maxJsonKeyLength to keep chunks below 48KB
nicktobey
committed
Detect and emit unknownLocationKeyError to more places.
nicktobey
committed
Disable chunk boundaries inside JSON string values, and emit largeJsonStringError for both keys and values in JSON.
nicktobey
committed
When encountering largeJsonStringError, fall back on inserting JSON document as a non-indexed BLOB
nicktobey
committed
Remove backwards compatibility tests. If we're emitting blobs, we know that older versions can read them.
nicktobey
committed
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
nicktobey
committed
Prevent an infinite loop when reading a JSON chunk that ends in the middle of a string.
nicktobey
committed
When encountering a jsonParseError when working with JSON, it's most likely a bug in handling JSON metadata. Emit a warning, then continue, treating the document as a blob and ignoring JSON metadata.
nicktobey
committed
Fix flipped sign in JsonCursor::NextValue
nicktobey
committed
Unskip dolt query test.
nicktobey
committed
Revert change to compatibility test.
nicktobey
committed
Update indexed document test.
nicktobey
committed