The AWS CloudTrail Processing Library is a Java client library that makes it easy to build an application that reads and processes CloudTrail log files in a fault tolerant and highly scalable manner.

• CloudTrail Product Page
• CloudTrail Forum
• Github Issues

• Provides functionality to continuously download CloudTrail log files in a fault tolerant and scalable manner.
• Serializes the events in JSON format to Plain Old Java Objects (POJO).
• Provides interfaces to implement your own business logic for selecting which events to process, processing events, handling errors, and handling log processing status updates.

• AWS Java SDK 1.12.261: To use the AWS CloudTrail Processing Library, you'll need the AWS Java SDK.
• Java 1.8: The AWS CloudTrail Processing Library requires Java 1.8 (Java SE 8) or later.

To learn how to use the AWS CloudTrail Processing Library to build a CloudTrail log processor in Java, read the documentation:

• Using the CloudTrail Processing Library in the AWS CloudTrail User Guide.
• AWS CloudTrail Processing Library Reference

After you've downloaded the code from GitHub, you can build it using Apache Maven. To disable GPG signing in the build, use this command:

mvn clean install -Dgpg.skip=true

Release 1.6.2 (Nov 30, 2023) ~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added support for modeling EdgeDeviceDetails

• Added support for test scenario for UserType IdentityCenterUser

• Updated CloudTrailEvent version to 1.12
• Added support for UserIdentity

• Update AWS Java SDK (S3/SQS) version to 1.12.261.

• Update AWS Java SDK (S3/SQS) version to 1.12.x

• Added support for implementing custom S3 manager.
• Added event logging to log file parsing-related exceptions.
• Added support for parsing optional errorCode field in insightDetails.
• Updated account ID parsing regex to accept non-numerical values.

• Added support for parsing the following new top-level optional fields:
  • addendum
  • edgeDeviceDetails
  • tlsDetails
  • sessionCredentialFromConsole
• Updated the CloudTrail event version to 1.08.

• Added support for parsing new section, attributions, in insightContext.
• Added support for parsing new fields, baselineDuration, in statistics section in insightContext.
• Added thread configuration for s3 client, sqs client, and sqs reader to enable performance tuning.
• Updated minimum required Java SE version to 1.8.

• Added support for a new eventCategory attribute to indicate whether an event is a management, data, or Insights event.
• Added support for Insights events, including new attributes like insightDetails or insightContext.
• Updated the CloudTrail event version to 1.07.

• Added support for automatically deleting the initial SNS validation message sent whenever an SNS topic for a trail is configured or updated. In previous releases, these messages had to be manually deleted.

• Patch Release 1.1.1

• Added support for Boolean identification of management events.
• Updated the CloudTrail event version to 1.06.

• Add support for different formats for SQS messages from the same SQS queue to identify CloudTrail log files. This includes the following:
  • Notifications that CloudTrail sends to an SNS topic.
  • Notifications that Amazon S3 sends to an SNS topic.
  • Notifications that Amazon S3 sends directly to the SQS queue.
• Add support for the new deleteMessageUponFailure property. Use this property to delete messages that the CloudTrail Processing Library can't process, such as the following:
  • Parsing message failure:
    • File is not JSON.
    • Notification is not an s3:ObjectCreated:Put event.
    • CloudTrail digest files, and other formats such as .jpeg or txt are unsupported.
  • Consuming log failure, such as processing events in a log file.

Note: If deleteMessageUponFailure is true, the CloudTrail Processing Library may delete messages that it can’t process. The default value is false. Learn more.

• Add support for ARN prefix to identify the ARNPrefix associated with the resource. Resource must have either ARN or ARNPrefix, but not both.
• Add support for shared event ID to identify CloudTrail events from the same AWS action that is sent to different AWS accounts.
• Add support for VPC endpoint ID to identify the VPC endpoint in which requests were made from a VPC to another AWS service, such as Amazon S3.
• Add support for annotation to identify user provided annotation tagging delivered by CloudTrail.
• Add support for identity provider to identify the principal name of the external identity provider.

• Add support for service event, additional information is provided in the serviceEventDetails file.
• Add support for Resource type to identify the resource's type in a given CloudTrail event.
• Update AWS Java SDK to version 1.11.
• Update the latest supported CloudTrail event version to 1.05.
• Update event version is not supported by CloudTrail warning logging message to debug level.

• Update AWS Java SDK to version 1.10.

• Initial release.