Components
Type: aws:core:s3PutListenerBoundary
Synopsis
A boundary that listens for S3 Put events and publishes an availability event for the dataset.
Description
This boundary listens for put events against the specified bucket and key prefix. If events arrive frequently in an interval, change the eventArrival to "frequent", this will cause the boundary to emit an availability event after the end of every interval regardless of available put events. If no put events are found, the dataset manifest status for the lot interval will be set to "empty". lotUnit: Fourths|Sixth|Twelfths|etc the interval of a lot, see documentation for supported intervals eventArrival: infrequent|frequent expected frequency of event arrivals infrequent is once per interval and frequent more than once per interval infrequent.lotSource: objectModifiedTime|eventTime objectModifiedTime will use the last modified time of the object eventTime will use the time the event was received by the listener infrequent.enableEventBridge: true|false true will enable event bridge on the bucket if the bucket was not declared with eventBridgeNotification enabled, it must be set here frequent.queueFetchWaitSec: seconds The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds. If no messages are available and the wait time expires, the call returns successfully with an empty list of messages. It is recommended to leave this value at zero (0). For frequently arriving events, all paths are collected until the end of the interval, except those paths that do not pass the filter, if given. The filter is a list of include and exclude patterns. Where: - ? matches one character - * matches zero or more characters - ** matches zero or more directories in a path A common exclude pattern would be '**/_*'. This would exclude all files that start with an underscore, like '_SUCCESS' or '_metadata'. frequent.filter.includes: A list of include patterns. Default is '**'. frequent.filter.excludes: A list of exclude patterns. frequent.filter.pathSeparator: The path separator to use when matching patterns. Default is '/'. frequent.filter.ignoreCase: Whether to ignore case when matching patterns. Default is false.
Template
Required
{
"type" : "aws:core:s3PutListenerBoundary",
"name" : null,
"dataset" : {
"name" : null,
"version" : null,
"pathURI" : null
},
"lotUnit" : null,
"eventArrival" : "infrequent"
}
All
{
"type" : "aws:core:s3PutListenerBoundary",
"name" : null,
"exclude" : false,
"dataset" : {
"name" : null,
"version" : null,
"pathURI" : null
},
"lotUnit" : null,
"eventArrival" : "infrequent",
"runtimeProps" : {
"memorySizeMB" : 1024,
"retryAttempts" : 3,
"timeoutMin" : 15,
"architecture" : "ARM_64"
},
"infrequent" : {
"lotSource" : "objectModifiedTime",
"enableEventBridge" : false
},
"frequent" : {
"queueFetchWaitSec" : 0
},
"filter" : {
"includes" : [ ],
"excludes" : [ ],
"pathSeparator" : "/",
"ignoreCase" : false
}
}