Data World 6 · Pipeline Detective · Stage 4

Rows Went Missing

Count the rows per day, then read the crawler log, the job runs, the quality results and the trail: each gap has one cause.

  • Crawler logs
  • Job run history
  • Bookmark resets
  • Archived partitions
  • Quality gates
Domain
Data engineering
World
Data World 6: Pipeline Detective (Associate)
Game type
Evidence desk: Each ticket is the reporter's words: find its cause in the evidence.
Time
About 25 minutes on a first attempt (estimated)
Exam guides
  • DEA-C01Skill 1.2.7: Troubleshoot and debug common transformation failures and performance issues; Skill 3.3.6: Troubleshoot and maintain pipelines (for example, AWS Glue, Amazon EMR); Skill 2.2.3: Discover schemas and use AWS Glue crawlers to populate data catalogs; Skill 2.4.2: Address changes to the characteristics of data; Skill 3.4.1: Run data quality checks while processing the data (for example, checking for empty fields)