AWS CloudTrail Processing Library

The AWS CloudTrail Processing Library is a Java client library that makes it easy to build an application that reads and processes CloudTrail log files in a fault tolerant and highly scalable manner.

• CloudTrail Product Page
• CloudTrail Forum
• Github Issues

Features

• Provides functionality to continuously download CloudTrail log files in a fault tolerant and scalable manner.
• Serializes the events in JSON format to Plain Old Java Objects (POJO).
• Provides interfaces to implement your own business logic for selecting which events to process, processing events, handling errors, and handling log processing status updates.

Getting Started

Minimum Requirements

• AWS Java SDK 1.10.27: In order to use the AWS CloudTrail Processing Library, you'll need the AWS Java SDK.
• Java 1.7: The AWS CloudTrail Processing Library requires Java 1.7 (Java SE 7) or later.

Documentation

To learn how to use the AWS CloudTrail Processing Library to build a CloudTrail log processor in Java, read the documentation:

• Using the CloudTrail Processing Library in the AWS CloudTrail User Guide.
• AWS CloudTrail Processing Library Reference

Building From Source

After you've downloaded the code from GitHub, you can build it using Apache Maven. To disable GPG signing in the build, use this command:

mvn clean install -Dgpg.skip=true

Release Notes

Release 1.1.2 (May 16, 2018)

• 1.1.2 includes the changes previously listed under 1.1.1. Namely:
  • Adds support for Boolean identification of management events.
  • Updates the CloudTrail event version to 1.06.

Release 1.1.1 (Nov 30, 2017) (Recalled)

• Changes originally intended for this version are now included in 1.1.2.

Release 1.1.0 (Jun 1, 2017)

• Add support for different formats for SQS messages from the same SQS queue to identify CloudTrail log files. This includes the following:
  • Notifications that CloudTrail sends to an SNS topic.
  • Notifications that Amazon S3 sends to an SNS topic.
  • Notifications that Amazon S3 sends directly to the SQS queue.
• Add support for the new deleteMessageUponFailure property. Use this property to delete messages that the CloudTrail Processing Library can't process, such as the following:
  • Parsing message failure:
    • File is not JSON.
    • Notification is not an s3:ObjectCreated:Put event.
    • CloudTrail digest files, and other formats such as .jpeg or txt are unsupported.
  • Consuming log failure, such as processing events in a log file.

Note: If deleteMessageUponFailure is true, the CloudTrail Processing Library may delete messages that it can’t process. The default value is false. Learn more.

Release 1.0.4 (Jan 17, 2017)

• Add support for ARN prefix to identify the ARNPrefix associated with the resource. Resource must have either ARN or ARNPrefix, but not both.
• Add support for shared event ID to identify CloudTrail events from the same AWS action that is sent to different AWS accounts.
• Add support for VPC endpoint ID to identify the VPC endpoint in which requests were made from a VPC to another AWS service, such as Amazon S3.
• Add support for annotation to identify user provided annotation tagging delivered by CloudTrail.
• Add support for identity provider to identify the principal name of the external identity provider.

Release 1.0.3 (Oct 5, 2016)

• Add support for service event, additional information is provided in the serviceEventDetails file.
• Add support for Resource type to identify the resource's type in a given CloudTrail event.
• Update AWS Java SDK to version 1.11.
• Update the latest supported CloudTrail event version to 1.05.
• Update event version is not supported by CloudTrail warning logging message to debug level.

Release 1.0.1 (Oct 28, 2015)

• Update AWS Java SDK to version 1.10.

Release 1.0.0 (Nov 3, 2014)

• Initial release.