Skip to content

Commit

Permalink
snappy: Update to 1.1.8.
Browse files Browse the repository at this point in the history
  • Loading branch information
jrfonseca committed Nov 21, 2020
1 parent 15de874 commit a9223f5
Show file tree
Hide file tree
Showing 14 changed files with 837 additions and 490 deletions.
8 changes: 8 additions & 0 deletions thirdparty/snappy/NEWS
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
Snappy v1.1.8, January 15th 2020:

* Small performance improvements.

* Removed snappy::string alias for std::string.

* Improved CMake configuration.

Snappy v1.1.7, August 24th 2017:

* Improved CMake build support for 64-bit Linux distributions.
Expand Down
39 changes: 19 additions & 20 deletions thirdparty/snappy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ In particular:

- Snappy uses 64-bit operations in several places to process more data at
once than would otherwise be possible.
- Snappy assumes unaligned 32- and 64-bit loads and stores are cheap.
- Snappy assumes unaligned 32 and 64-bit loads and stores are cheap.
On some platforms, these must be emulated with single-byte loads
and stores, which is much slower.
- Snappy assumes little-endian throughout, and needs to byte-swap data in
Expand All @@ -65,32 +65,37 @@ are of course most welcome; see "Contact", below.
Building
========

CMake is supported and autotools will soon be deprecated.
You need CMake 3.4 or above to build:

mkdir build
cd build && cmake ../ && make
You need the CMake version specified in [CMakeLists.txt](./CMakeLists.txt)
or later to build:

```bash
mkdir build
cd build && cmake ../ && make
```

Usage
=====

Note that Snappy, both the implementation and the main interface,
is written in C++. However, several third-party bindings to other languages
are available; see the home page at http://google.github.io/snappy/
for more information. Also, if you want to use Snappy from C code, you can
use the included C bindings in snappy-c.h.
are available; see the [home page](docs/README.md) for more information.
Also, if you want to use Snappy from C code, you can use the included C
bindings in snappy-c.h.

To use Snappy from your own C++ program, include the file "snappy.h" from
your calling file, and link against the compiled library.

There are many ways to call Snappy, but the simplest possible is

snappy::Compress(input.data(), input.size(), &output);
```c++
snappy::Compress(input.data(), input.size(), &output);
```
and similarly
snappy::Uncompress(input.data(), input.size(), &output);
```c++
snappy::Uncompress(input.data(), input.size(), &output);
```

where "input" and "output" are both instances of std::string.

Expand All @@ -112,12 +117,12 @@ tests to verify you have not broken anything. Note that if you have the
Google Test library installed, unit test behavior (especially failures) will be
significantly more user-friendly. You can find Google Test at

http://github.com/google/googletest
https://github.com/google/googletest

You probably also want the gflags library for handling of command-line flags;
you can find it at

http://gflags.github.io/gflags/
https://gflags.github.io/gflags/

In addition to the unit tests, snappy contains microbenchmarks used to
tune compression and decompression performance. These are automatically run
Expand All @@ -140,10 +145,4 @@ Contact
=======

Snappy is distributed through GitHub. For the latest version, a bug tracker,
and other information, see

http://google.github.io/snappy/

or the repository at

https://github.com/google/snappy
and other information, see https://github.com/google/snappy.
13 changes: 6 additions & 7 deletions thirdparty/snappy/config/config.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,6 @@
/* Define to 1 if you have the `z' library (-lz). */
/* #undef HAVE_LIBZ */

/* Define to 1 if you have the <stddef.h> header file. */
#define HAVE_STDDEF_H 1

/* Define to 1 if you have the <stdint.h> header file. */
#define HAVE_STDINT_H 1

/* Define to 1 if you have the <stdlib.h> header file. */
/* Define to 1 if you have the <sys/endian.h> header file. */
/* #undef HAVE_SYS_ENDIAN_H */

Expand All @@ -69,6 +62,12 @@
# define HAVE_WINDOWS_H 1
#endif

/* Define to 1 if you target processors with SSSE3+ and have <tmmintrin.h>. */
/* #undef SNAPPY_HAVE_SSSE3 */

/* Define to 1 if you target processors with BMI2+ and have <bmi2intrin.h>. */
/* #undef SNAPPY_HAVE_BMI2 */

/* Define to 1 if your processor stores words with the most significant byte
first (like Motorola and SPARC, unlike Intel and VAX). */
#if defined(__GLIBC__)
Expand Down
17 changes: 12 additions & 5 deletions thirdparty/snappy/snappy-internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,19 +36,26 @@
namespace snappy {
namespace internal {

// Working memory performs a single allocation to hold all scratch space
// required for compression.
class WorkingMemory {
public:
WorkingMemory() : large_table_(NULL) { }
~WorkingMemory() { delete[] large_table_; }
explicit WorkingMemory(size_t input_size);
~WorkingMemory();

// Allocates and clears a hash table using memory in "*this",
// stores the number of buckets in "*table_size" and returns a pointer to
// the base of the hash table.
uint16* GetHashTable(size_t input_size, int* table_size);
uint16* GetHashTable(size_t fragment_size, int* table_size) const;
char* GetScratchInput() const { return input_; }
char* GetScratchOutput() const { return output_; }

private:
uint16 small_table_[1<<10]; // 2KB
uint16* large_table_; // Allocated only when needed
char* mem_; // the allocated memory, never nullptr
size_t size_; // the size of the allocated memory, never 0
uint16* table_; // the pointer to the hashtable
char* input_; // the pointer to the input scratch buffer
char* output_; // the pointer to the output scratch buffer

// No copying
WorkingMemory(const WorkingMemory&);
Expand Down
2 changes: 1 addition & 1 deletion thirdparty/snappy/snappy-stubs-internal.cc
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@

namespace snappy {

void Varint::Append32(string* s, uint32 value) {
void Varint::Append32(std::string* s, uint32 value) {
char buf[Varint::kMax32];
const char* p = Varint::Encode32(buf, value);
s->append(buf, p - buf);
Expand Down
75 changes: 60 additions & 15 deletions thirdparty/snappy/snappy-stubs-internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,18 @@
#include <intrin.h>
#endif // defined(_MSC_VER)

#ifndef __has_feature
#define __has_feature(x) 0
#endif

#if __has_feature(memory_sanitizer)
#include <sanitizer/msan_interface.h>
#define SNAPPY_ANNOTATE_MEMORY_IS_INITIALIZED(address, size) \
__msan_unpoison((address), (size))
#else
#define SNAPPY_ANNOTATE_MEMORY_IS_INITIALIZED(address, size) /* empty */
#endif // __has_feature(memory_sanitizer)

#include "snappy-stubs-public.h"

#if defined(__x86_64__)
Expand Down Expand Up @@ -187,7 +199,7 @@ struct Unaligned32Struct {
((reinterpret_cast< ::snappy::base::internal::Unaligned32Struct *>(_p))->value = \
(_val))

// TODO(user): NEON supports unaligned 64-bit loads and stores.
// TODO: NEON supports unaligned 64-bit loads and stores.
// See if that would be more efficient on platforms supporting it,
// at least for copies.

Expand Down Expand Up @@ -353,6 +365,9 @@ class LittleEndian {
// Some bit-manipulation functions.
class Bits {
public:
// Return floor(log2(n)) for positive integer n.
static int Log2FloorNonZero(uint32 n);

// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0.
static int Log2Floor(uint32 n);

Expand All @@ -373,50 +388,72 @@ class Bits {

#ifdef HAVE_BUILTIN_CTZ

inline int Bits::Log2FloorNonZero(uint32 n) {
assert(n != 0);
// (31 ^ x) is equivalent to (31 - x) for x in [0, 31]. An easy proof
// represents subtraction in base 2 and observes that there's no carry.
//
// GCC and Clang represent __builtin_clz on x86 as 31 ^ _bit_scan_reverse(x).
// Using "31 ^" here instead of "31 -" allows the optimizer to strip the
// function body down to _bit_scan_reverse(x).
return 31 ^ __builtin_clz(n);
}

inline int Bits::Log2Floor(uint32 n) {
return n == 0 ? -1 : 31 ^ __builtin_clz(n);
return (n == 0) ? -1 : Bits::Log2FloorNonZero(n);
}

inline int Bits::FindLSBSetNonZero(uint32 n) {
assert(n != 0);
return __builtin_ctz(n);
}

#if defined(ARCH_K8) || defined(ARCH_PPC) || defined(ARCH_ARM)
inline int Bits::FindLSBSetNonZero64(uint64 n) {
assert(n != 0);
return __builtin_ctzll(n);
}
#endif // defined(ARCH_K8) || defined(ARCH_PPC) || defined(ARCH_ARM)

#elif defined(_MSC_VER)

inline int Bits::Log2FloorNonZero(uint32 n) {
assert(n != 0);
unsigned long where;
_BitScanReverse(&where, n);
return static_cast<int>(where);
}

inline int Bits::Log2Floor(uint32 n) {
unsigned long where;
if (_BitScanReverse(&where, n)) {
return where;
} else {
return -1;
}
if (_BitScanReverse(&where, n))
return static_cast<int>(where);
return -1;
}

inline int Bits::FindLSBSetNonZero(uint32 n) {
assert(n != 0);
unsigned long where;
if (_BitScanForward(&where, n)) return static_cast<int>(where);
if (_BitScanForward(&where, n))
return static_cast<int>(where);
return 32;
}

#if defined(ARCH_K8) || defined(ARCH_PPC) || defined(ARCH_ARM)
inline int Bits::FindLSBSetNonZero64(uint64 n) {
assert(n != 0);
unsigned long where;
if (_BitScanForward64(&where, n)) return static_cast<int>(where);
if (_BitScanForward64(&where, n))
return static_cast<int>(where);
return 64;
}
#endif // defined(ARCH_K8) || defined(ARCH_PPC) || defined(ARCH_ARM)

#else // Portable versions.

inline int Bits::Log2Floor(uint32 n) {
if (n == 0)
return -1;
inline int Bits::Log2FloorNonZero(uint32 n) {
assert(n != 0);

int log = 0;
uint32 value = n;
for (int i = 4; i >= 0; --i) {
Expand All @@ -431,7 +468,13 @@ inline int Bits::Log2Floor(uint32 n) {
return log;
}

inline int Bits::Log2Floor(uint32 n) {
return (n == 0) ? -1 : Bits::Log2FloorNonZero(n);
}

inline int Bits::FindLSBSetNonZero(uint32 n) {
assert(n != 0);

int rc = 31;
for (int i = 4, shift = 1 << 4; i >= 0; --i) {
const uint32 x = n << shift;
Expand All @@ -447,6 +490,8 @@ inline int Bits::FindLSBSetNonZero(uint32 n) {
#if defined(ARCH_K8) || defined(ARCH_PPC) || defined(ARCH_ARM)
// FindLSBSetNonZero64() is defined in terms of FindLSBSetNonZero().
inline int Bits::FindLSBSetNonZero64(uint64 n) {
assert(n != 0);

const uint32 bottombits = static_cast<uint32>(n);
if (bottombits == 0) {
// Bottom bits are zero, so scan in top bits
Expand Down Expand Up @@ -479,7 +524,7 @@ class Varint {
static char* Encode32(char* ptr, uint32 v);

// EFFECTS Appends the varint representation of "value" to "*s".
static void Append32(string* s, uint32 value);
static void Append32(std::string* s, uint32 value);
};

inline const char* Varint::Parse32WithLimit(const char* p,
Expand Down Expand Up @@ -536,7 +581,7 @@ inline char* Varint::Encode32(char* sptr, uint32 v) {
// replace this function with one that resizes the string without
// filling the new space with zeros (if applicable) --
// it will be non-portable but faster.
inline void STLStringResizeUninitialized(string* s, size_t new_size) {
inline void STLStringResizeUninitialized(std::string* s, size_t new_size) {
s->resize(new_size);
}

Expand All @@ -552,7 +597,7 @@ inline void STLStringResizeUninitialized(string* s, size_t new_size) {
// (http://www.open-std.org/JTC1/SC22/WG21/docs/lwg-defects.html#530)
// proposes this as the method. It will officially be part of the standard
// for C++0x. This should already work on all current implementations.
inline char* string_as_array(string* str) {
inline char* string_as_array(std::string* str) {
return str->empty() ? NULL : &*str->begin();
}

Expand Down
Loading

0 comments on commit a9223f5

Please sign in to comment.