BadgerDB v3.2103.3

This is a minor patch release that fixes arm64 related issues. The issues in the z package in Ristretto were resolved in Ristretto v0.1.1.

Fixed

• fix(arm64): bump ristretto v0.1.0 --> v0.1.1 (#1806)

Full Changelog: v3.2103.2...v3.2103.3

BadgerDB v3.2103.2

This patch release contains:

Fixed

• fix(compact): close vlog after the compaction at L0 has been completed (#1752)
• fix(builder): put the upper limit on reallocation (#1748)
• deps: Bump github.com/google/flatbuffers to v1.12.1 (#1746)
• fix(levels): Avoid a deadlock when acquiring read locks in levels (#1744)
• fix(pubsub): avoid deadlock in publisher and subscriber (#1749) (#1751)

Full Changelog: v3.2103.1...v3.2103.2

BadgerDB v2.2007.4

Fixed

• Fix build on Plan 9 (#1451) (#1508) (#1738)

Features

• feat(zstd): backport replacement of DataDog's zstd with Klauspost's zstd (#1736)

BadgerDB v2.2007.3

This patch release contains:

Fixed

• fix(maxVersion): Use choosekey instead of KeyToList (#1532) #1533
• fix(flatten): Add --num_versions flag (#1518) #1520
• fix(build): Fix integer overflow on 32-bit architectures #1558
• fix(pb): avoid protobuf warning due to common filename (#1519)

Features

• Add command to stream contents of DB into another DB. (#1486)

New APIs

• DB.StreamDB
• DB.MaxVersion

BadgerDB v3.2103.1

This release removes CGO dependency opf badger by using Klauspost's ZSTD instead of Datadog's ZSTD. Also, this has some of the fixes.

Fixed

• fix(compaction): copy over the file ID when building tables #1713
• fix: Fix conflict detection for managed DB (#1716)
• fix(pendingWrites): don't skip the pending entries with version=0 (#1721)

Features

• feat(zstd): replace datadog's zstd with Klauspost's zstd (#1709)

BadgerDB v3.2103.0

Breaking

• Subscribe: Add option to subscribe with holes in prefixes. (#1658)

Fixed

• fix(compaction): Remove compaction backoff mechanism (#1686)
• Add a name to mutexes to make them unexported (#1678)
• fix(merge-operator): don't read the deleted keys (#1675)
• fix(discard): close the discard stats file on db close (#1672)
• fix(iterator): fix iterator when data does not exist in read only mode (#1670)
• fix(badger): Do not reuse variable across badger commands (#1624)
• fix(dropPrefix): check properly if the key is present in a table (#1623)

Performance

• Opt(Stream): Optimize how we deduce key ranges for iteration (#1687)
• Increase value threshold from 1 KB to 1 MB (#1664)
• opt(DropPrefix): check if there exist some data to drop before dropping prefixes (#1621)

Features

• feat(options): allow special handling and checking when creating options from superflag (#1688)
• overwrite default Options from SuperFlag string (#1663)
• Support SinceTs in iterators (#1653)
• feat(info): Add a flag to parse and print DISCARD file (#1662)
• feat(vlog): making vlog threshold dynamic 6ce3b7c (#1635)
• feat(options): add NumGoroutines option for default Stream.numGo (#1656)
• feat(Trie): Working prefix match with holes (#1654)
• feat: add functionality to ban a prefix (#1638)
• feat(compaction): Support Lmax to Lmax compaction (#1615)

New APIs

• Badger.DB
  • BanNamespace
  • BannedNamespaces
  • Ranges
• Badger.Options
  • FromSuperFlag
  • WithNumGoRoutines
  • WithNamespaceOffset
  • WithVLogPercentile
• Badger.Trie
  • AddMatch
  • DeleteMatch
• Badger.Table
  • StaleDataSize
• Badger.Table.Builder
  • AddStaleKey
• Badger.InitDiscardStats

Removed APIs

• Badger.DB
  • KeySplits
• Badger.Options
  • SkipVlog

Changed APIs

• Badger.DB
  • Subscribe
• Badger.Options
  • WithValueThreshold

BadgerDB v3.2011.1

fix(compaction): Set base level correctly after stream (#1631) (#1651)
fix: update ristretto and use filepath (#1649) (#1652)
fix(badger): Do not reuse variable across badger commands (#1624) (#1650)
fix(build): fix 32-bit build (#1627) (#1646)
fix(table): always sync SST to disk (#1625) (#1645)

BadgerDB v3.2011.0

This release is not backward compatible with Badger v2.x.x

Breaking:

• opt(compactions): Improve compaction performance (#1574)
• Change how Badger handles WAL (#1555)
• feat(index): Use flatbuffers instead of protobuf (#1546)

Fixed:

• Fix(GC): Set bits correctly for moved keys (#1619)
• Fix(tableBuilding): reduce scope of valuePointer (#1617)
• Fix(compaction): fix table size estimation on compaction (#1613)
• Fix(OOM): Reuse pb.KVs in Stream (#1609)
• Fix race condition in L0StallMs variable (#1605)
• Fix(stream): Stop produceKVs on error (#1604)
• Fix(skiplist): Remove z.Buffer from skiplist (#1600)
• Fix(readonly): fix the file opening mode (#1592)
• Fix: Disable CompactL0OnClose by default (#1586)
• Fix(compaction): Don't drop data when split overlaps with top tables (#1587)
• Fix(subcompaction): Close builder before throttle.Done (#1582)
• Fix(table): Add onDisk size (#1569)
• Fix(Stream): Only send done markers if told to do so
• Fix(value log GC): Fix a bug which caused value log files to not be GCed.
• Fix segmentation fault when cache sizes are small. (#1552)
• Fix(builder): Too many small tables when compression is enabled (#1549)
• Fix integer overflow error when building for 386 (#1541)
• Fix(writeBatch): Avoid deadlock in commit callback (#1529)
• Fix(db): Handle nil logger (#1534)
• Fix(maxVersion): Use choosekey instead of KeyToList (#1532)
• Fix(Backup/Restore): Keep all versions (#1462)
• Fix(build): Fix nocgo builds. (#1493)
• Fix(cleanup): Avoid truncating in value.Open on error (#1465)
• Fix(compaction): Don't use cache for table compaction (#1467)
• Fix(compaction): Use separate compactors for L0, L1 (#1466)
• Fix(options): Do not implicitly enable cache (#1458)
• Fix(cleanup): Do not close cache before compaction (#1464)
• Fix(replay): Update head for LSM entires also (#1456)
• fix(levels): Cleanup builder resources on building an empty table (#1414)

Performance

• perf(GC): Remove move keys (#1539)
• Keep the cheaper parts of the index within table struct. (#1608)
• Opt(stream): Use z.Buffer to stream data (#1606)
• opt(builder): Use z.Allocator for building tables (#1576)
• opt(memory): Use z.Calloc for allocating KVList (#1563)
• opt: Small memory usage optimizations (#1562)
• KeySplits checks tables and memtables when number of splits is small. (#1544)
• perf: Reduce memory usage by better struct packing (#1528)
• perf(tableIterator): Don't do next on NewIterator (#1512)
• Improvements: Manual Memory allocation via Calloc (#1459)
• Various bug fixes: Break up list and run DropAll func (#1439)
• Add a limit to the size of the batches sent over a stream. (#1412)
• Commit does not panic after Finish, instead returns an error (#1396)
• levels: Compaction incorrectly drops some delete markers (#1422)
• Remove vlog file if bootstrap, syncDir or mmap fails (#1434)

Features:

• Use opencensus for tracing (#1566)
• Export functions from Key Registry (#1561)
• Allow sizes of block and index caches to be updated. (#1551)
• Add metric for number of tables being compacted (#1554)
• feat(info): Show index and bloom filter size (#1543)
• feat(db): Add db.MaxVersion API (#1526)
• Expose DB options in Badger. (#1521)
• Feature: Add a Calloc based Buffer (#1471)
• Add command to stream contents of DB into another DB. (#1463)
• Expose NumAlloc metrics via expvar (#1470)
• Support fully disabling the bloom filter (#1319)
• Add --enc-key flag in badger info tool (#1441)

New APIs

• Badger.DB
  • CacheMaxCost (#1551)
  • Levels (#1574)
  • LevelsToString (#1574)
  • Opts (#1521)
• Badger.Options
  • WithBaseLevelSize (#1574)
  • WithBaseTableSize (#1574)
  • WithMemTableSize (#1574)
• Badger.KeyRegistry
  • DataKey (#1561)
  • LatestDataKey (#1561)

Removed APIs

• Badger.Options
  • WithKeepL0InMemory (#1555)
  • WithLevelOneSize (#1574)
  • WithLoadBloomsOnOpen (#1555)
  • WithLogRotatesToFlush (#1574)
  • WithMaxTableSize (#1574)
  • WithTableLoadingMode (#1555)
  • WithTruncate (#1555)
  • WithValueLogLoadingMode (#1555)

BadgerDB v1.6.2

Fixed

• Fix Sequence generates duplicate values (#1281)
• Ensure bitValuePointer flag is cleared for LSM entry values written to LSM (#1313)
• Confirm badgerMove entry required before rewrite (#1302)
• Drop move keys when its key prefix is dropped (#1331)
• Compaction: Expired keys and delete markers are never purged (#1354)
• Restore: Account for value size as well (#1358)
• GC: Consider size of value while rewriting (#1357)
• Rework DB.DropPrefix (#1381)
• Update head while replaying value log (#1372)
• Remove vlog file if bootstrap, syncDir or mmap fails (#1434)
• Levels: Compaction incorrectly drops some delete markers (#1422)
• Fix(replay) - Update head for LSM entries also (#1456)
• Fix(Backup/Restore): Keep all versions (#1462)
• Fix build on Plan 9 (#1451)

BadgerDB v2.2007.2

Fixed

• Compaction: Use separate compactors for L0, L1 (#1466)
• Rework Block and Index cache (#1473)
• Add IsClosed method (#1478)
• Cleanup: Avoid truncating in vlog.Open on error (#1465)
• Cleanup: Do not close cache before compactions (#1464)

New APIs

• Badger.DB
  • BlockCacheMetrics (#1473)
  • IndexCacheMetrics (#1473)
• Badger.Option
  • WithBlockCacheSize (#1473)
  • WithIndexCacheSize (#1473)

Removed APIs [Breaking Changes]

• Badger.DB
  • DataCacheMetrics (#1473)
  • BfCacheMetrics (#1473)
• Badger.Option
  • WithMaxCacheSize (#1473)
  • WithMaxBfCacheSize (#1473)
  • WithKeepBlockIndicesInCache (#1473)
  • WithKeepBlocksInCache (#1473)