Release 0.14.7 of the Amazon Kinesis Producer Library

Release Notes

Java

0.14.7

• PR #350 Upgrading Guava to 29.0-jre
• PR #352 Upgrading Commons IO to 2.7
• PR #351 Adding support for proxy configurations
• PR #356 Fixing build issues in Travis CI

Release 0.14.6 of the Amazon Kinesis Producer Library

Release 0.14.6 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

Java

• [PR #341] Updating Java SDK version in KPL to 1.11.960

Release 0.14.5 of the Amazon Kinesis Producer Library

Release 0.14.5 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

C++ Core

• [PR #339] Fixing KPL not emitting Kinesis PutRecords call context metrics.

Release 0.14.4 of the Amazon Kinesis Producer Library

Release 0.14.4 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

Java

• [PR #334] Add support for building multiple architectures, specifically arm64.
  • This now supports AWS Graviton based instances.
  • Bumped Boost slightly to a version that includes Arm support and added the architecture to the path for kinesis_producer.
• [PR #335] Fixed logging for native layer allowing to enable debug/trace logs.

C++ Core

• [PR #335] Fixed logging for native layer allowing to enable debug/trace logs.

Release 0.14.3 of the Amazon Kinesis Producer Library

Release 0.14.3 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

Java

• [PR #327] Adding support for timeout on user records at Java layer.
  • New optional KPL config parameter userRecordTimeoutInMillis which can be used to timeout records at the java layer queued for processing.
• [PR #324] Adding KPL metric to track the time for oldest user record in processing at the java layer.

C++ Core

• [PR #328] Changing CloudWatch client retry strategy to use default SDK retry strategy with exponential backoff.
• [PR #318] Fixing bug where KPL goes into a continuous retry storm if the stream is deleted and re-created.

Release 0.14.2 of the Amazon Kinesis Producer Library

Release 0.14.2 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

Java

• [PR #320] Adding support for Glue Schema Registry.
  Serialize and send schemas along with records, support for compression and auto-registration of schemas.
• [PR #316] Bumping junit from 4.12 to 4.13.1
• [PR #312] Adding new parameter in KPL config to allow cert path to be overridden.
• [PR #310] Fixing bug to make the executor service to use 4*num_cores threads.
• [PR #307] Dependency Upgrade
  Upgrade Guava to 26.0-jre
• Update BOOST C++ Libraries link as cert expired on the older link

Release 0.14.1 of the Amazon Kinesis Producer Library

Release 0.14.1 of the Amazon Kinesis Producer Library

Amazon Kinesis Producer Library

Java

• [PR #302] Dependency Upgrade
  • upgrade org.hibernate.validator:hibernate-validator 6.0.2.Final -> 6.0.18.Final
  • upgrade com.google.guava:guava 18.0 -> 24.1.1-jre
• [PR #300] Fix Travis CI build issues
• [PR #298] Upgrade google-protobuf to 3.11.4

Release 0.14.0 of the Amazon Kinesis Producer Library

Required KPL Update – v0.14.0

KPL 0.14.0 now uses ListShards API, making it easier for your Kinesis Producer applications to scale. Kinesis Data Streams (KDS) enables you to scale your stream capacity without any changes to producers and consumers. After a scaling event, producer applications need to discover the new shard map. Version 0.14.0 replaces the DescribeStream with the ListShards API for shard discovery. ListShards API supports 100TPS per stream compared to DescribeStream that supports 10TPS per account z. For an account with 10 streams using KPL v0.14.0 will provide you a 100X higher call rate for shard discovery, eliminating the need for a DescribeStream API limit increase for scaling. You can find more information on the ListShards API in the Kinesis Data Streams documentation.

Recommended Settings for Streams larger than 800 shards

The KPL is an application for ingesting data to your Kinesis Data Streams. As your streams grow you may find the need to tune the KPL unable to accommodate the growing needs of your applications. Without optimized configurations your KPL processes will see inefficient CPU usage and delays in writing records into KDS. For streams larger than 800 shards, we recommend the following settings:

• ThreadingModel= “POOLED”
• MetricsGranularity= “stream”
• ThreadPoolSize=128

We recommend performing sufficient testing before applying these changes to production, as every customer has different usage patterns

Release Notes

0.14.0

• Note: Windows platform will be unsupported going forward for this library.
• [PR #280] When aggregation is enabled and all the buffer time is consumed for aggregating User records into Kinesis records, allow some additional buffer time for aggregating Kinesis Records into PutRecords calls.
• [PR #260] Added endpoint for China Ningxia region (cn-northwest-1).
• [PR #277] Changed mechanism to update the shard map
  • Switched to using ListShards instead of DescribeStream, as this is a more scalable API
  • Reduced the number of unnecessary shard map invalidations
  • Reduced the number of unnecessary update shard map calls
  • Reduced logging noise for aggregated records landing on an unexpected shard
• [PR #276] Updated AWS SDK from 1.0.5 to 1.7.180
• [PR #275] Improved the sample code to avoid need to edit code to run.
• [PR #274] Updated bootstrap.sh to build all dependencies and pack binaries into the jar.
• [PR #273] Added compile flags to enable compiling aws-sdk-cpp with Gcc7.
• [PR #229] Fixed bootstrap.sh to download dependent libraries directly from source.
• [PR #246] [PR #264] Various Typos

Release 0.13.1 of the Amazon Kinesis Producer Library

Releasing the windows artifacts along with other Platforms for Apache 2.0 license change.

Java

• [PR #256] Update KPL to Apache 2.0
• [PR #267] Prepare for windows release by increasing version number.

C++ Core

• [PR #256] Update KPL to Apache 2.0

Release 0.13.0 of the Amazon Kinesis Producer Library

Java

• [PR #256] Update KPL to Apache 2.0

C++ Core

• [PR #256] Update KPL to Apache 2.0