Data World 2 · Pipeline Plumbing · Stage 2

On Arrival

A trigger runs for every object it matches, once per event, one configuration per bucket: match the data, skip the rest, and batch what arrives every few minutes.

S3 Event Notifications
Amazon EventBridge
Lambda recursive loops
Glue job concurrency

Domain: Data engineering
World: Data World 2: Pipeline Plumbing (Associate)
Game type: Lake tuner: Tune a lake's tables until the queries fit their limit and budget.
Time: About 25 minutes on a first attempt (estimated)
Exam guides: DEA-C01Skill 1.1.6: Set up event triggers (for example, Amazon S3 Event Notifications, EventBridge); Skill 1.1.5: Set up schedulers by using Amazon EventBridge, Apache Airflow, or time-based schedules for jobs and crawlers; Skill 3.1.9: Manage events and schedulers (for example, Amazon EventBridge); Skill 3.1.8: Use AWS Lambda to automate data processing; Skill 1.3.1: Use orchestration services to build workflows for data ETL pipelines (for example, Lambda, EventBridge, Amazon Managed Workflows for Apache Airflow [Amazon MWAA], AWS Step Functions, AWS Glue workflows)