AWS · Certification path · DEA-C01
DEA-C01: Data Engineer - Associate
56 stages in study order, about 23 hours to complete, drawn from Networking, AI and machine learning, Data engineering, Security and Core architecture. Each stage names the section of the DEA-C01 exam guide it practises.
The path, in study order
- 1Shape the FilesFreeData engineering · Data World 1 · Skill 1.2.6, Skill 2.4.5, Skill 1.2.4, Skill 3.1.7
- 2Cut by DateData engineering · Data World 1 · Skill 2.4.5, Skill 1.2.4, Skill 1.2.6, Skill 3.1.7
- 3Find the SchemaData engineering · Data World 1 · Skill 2.2.2, Skill 2.2.3, Skill 2.4.2, Skill 1.2.3
- 4Register the DayData engineering · Data World 1 · Skill 2.2.4, Skill 1.1.5, Skill 2.2.1, Skill 3.1.7
- 5Keep, Tier, ExpireData engineering · Data World 1 · Skill 2.3.2, Skill 2.3.3, Skill 2.3.4, Skill 2.3.5
- 6Open TablesData engineering · Data World 1 · Skill 2.1.7, Skill 2.3.5
- 7Right PipeData engineering · Data World 2 · Skill 1.1.2, Skill 1.1.1, Skill 1.1.4, Skill 2.1.4
- 8On ArrivalData engineering · Data World 2 · Skill 1.1.6, Skill 1.1.5, Skill 3.1.9, Skill 3.1.8, Skill 1.3.1
- 9Inside the FenceData engineering · Data World 2 · Skill 1.2.2, Skill 2.2.5, Skill 1.1.8, Skill 4.1.1, Skill 4.1.5, Skill 4.2.2
- 10Only the NewData engineering · Data World 2 · Skill 1.1.11, Skill 1.1.12, Skill 1.2.7, Skill 3.3.6
- 11Conduct the RunData engineering · Data World 2 · Skill 1.3.1, Skill 1.3.2, Skill 1.3.3, Skill 1.3.4, Skill 3.1.1, Skill 3.1.2
- 12Change CaptureData engineering · Data World 2 · Skill 1.1.1, Skill 1.1.2, Skill 2.4.3
- 13Pick the StreamData engineering · Data World 3 · Skill 1.1.1, Skill 1.1.10, Skill 1.1.11, Skill 1.2.9, Skill 2.1.1
- 14Hot ShardData engineering · Data World 3 · Skill 1.1.9, Skill 2.1.1, Skill 1.1.1
- 15Many ReadersData engineering · Data World 3 · Skill 1.1.10, Skill 1.1.11, Skill 1.1.1
- 16Poison PillData engineering · Data World 3 · Skill 1.1.7, Skill 1.4.2, Skill 3.1.8, Skill 1.3.2
- 17Land It in S3Data engineering · Data World 3 · Skill 1.2.6, Skill 2.4.5, Skill 1.2.7, Skill 1.1.1
- 18Queue ItData engineering · Data World 3 · Skill 1.4.2, Skill 1.3.2, Skill 3.1.8
- 19Who Holds the KeysData engineering · Data World 4 · Skill 4.2.4, Skill 4.2.6, Skill 4.1.2
- 20Columns and RowsData engineering · Data World 4 · Skill 4.2.4, Skill 4.2.5, Skill 4.2.6
- 21Tag the LakeData engineering · Data World 4 · Skill 4.2.5, Skill 4.2.4, Skill 4.5.7
- 22Share AcrossData engineering · Data World 4 · Skill 4.5.1, Skill 4.3.3, Skill 2.2.6, Skill 4.5.6, Skill 4.1.7
- 23Lock the DataData engineering · Data World 4 · Skill 4.3.2, Skill 4.3.4, Skill 4.1.4, Skill 4.2.1
- 24Find the PIIData engineering · Data World 4 · Skill 4.5.2, Skill 4.3.1, Skill 4.4.1, Skill 4.4.3, Skill 4.5.3, Skill 4.5.4, Skill 4.5.5
- 25Pick the StoreData engineering · Data World 5 · Skill 2.1.1, Skill 2.1.2, Skill 2.1.3, Skill 2.1.8
- 26Load It RightData engineering · Data World 5 · Skill 2.3.1, Skill 1.1.3, Skill 1.1.2, Skill 2.3.6, Skill 1.1.9
- 27Keys to the JoinData engineering · Data World 5 · Skill 2.4.1, Skill 2.4.5, Skill 3.3.4, Skill 3.2.3, Skill 2.1.6
- 28Query in PlaceData engineering · Data World 5 · Skill 2.1.5, Skill 3.2.3, Skill 3.2.5, Skill 1.2.3, Skill 1.1.1, Skill 3.2.1, Skill 3.2.4, Skill 3.2.6, Skill 1.2.8
- 29Share the WarehouseData engineering · Data World 5 · Skill 4.5.1, Skill 4.2.3, Skill 4.2.5, Skill 4.3.1, Skill 4.3.3, Skill 4.5.7, Skill 2.1.6, Skill 2.4.4
- 30Right Engine, Right PriceData engineering · Data World 5 · Skill 1.2.5, Skill 1.2.4, Skill 1.2.1, Skill 3.1.4, Skill 3.2.5, Skill 1.2.10
- 31Quality GateData engineering · Data World 6 · Skill 3.4.1, Skill 3.4.2, Skill 3.4.3, Skill 3.4.4, Skill 3.4.5, Skill 3.2.2, Skill 3.1.6
- 32Watch the PipesData engineering · Data World 6 · Skill 3.3.2, Skill 3.3.3, Skill 3.3.7, Skill 3.3.4, Skill 1.3.4, Skill 3.1.9
- 33The Bill DoubledData engineering · Data World 6 · Skill 1.2.4, Skill 3.3.4, Skill 3.2.5, Skill 3.1.7
- 34Rows Went MissingData engineering · Data World 6 · Skill 1.2.7, Skill 3.3.6, Skill 2.2.3, Skill 2.4.2, Skill 3.4.1
- 35Stream Is LaggingData engineering · Data World 6 · Skill 1.1.7, Skill 1.1.9, Skill 1.1.10, Skill 1.4.2, Skill 3.3.4, Skill 3.3.8
- 36Who Touched the DataData engineering · Data World 6 · Skill 3.3.5, Skill 3.3.1, Skill 3.3.8, Skill 4.4.1, Skill 4.4.2, Skill 4.4.3, Skill 4.4.4, Skill 4.4.5, Skill 4.5.4, Skill 2.4.4
- 37Guard RailsNetworking · World 1 · Skill 4.1.1
- 38Shortcut to S3Networking · World 1 · Skill 4.1.5
- 39Cut and EmbedAI and machine learning · AI World 2 · Skill 2.4.6
- 40Pick the StoreAI and machine learning · AI World 2 · Skill 2.1.3
- 41Deny WinsFreeSecurity · Security World 1 · Skill 4.2.1
- 42Trust Both WaysSecurity · Security World 1 · Skill 4.3.3
- 43Inside the BoundarySecurity · Security World 1 · Skill 4.2.6
- 44Fence the OrgSecurity · Security World 2 · Skill 4.5.3
- 45Key Policy FirstSecurity · Security World 3 · Skill 4.3.2, Skill 4.3.3
- 46At Rest EverywhereSecurity · Security World 3 · Skill 4.3.2
- 47On the WireSecurity · Security World 3 · Skill 4.3.4
- 48Secrets and MasksSecurity · Security World 3 · Skill 4.1.3, Skill 4.2.2
- 49Log What MattersSecurity · Security World 5 · Skill 4.4.1
- 50Lake of LogsSecurity · Security World 5 · Skill 4.4.3
- 51Exposed KeySecurity · Security World 6 · Skill 4.1.3
- 52Trace the RootSecurity · Security World 6 · Skill 4.4.3
- 53Objects at ScaleCore architecture · Architecture World 1 · Skill 2.3.6
- 54Capacity UnitsCore architecture · Architecture World 2 · Skill 1.1.9
- 55Pick the PlatformCore architecture · Architecture World 4 · Skill 1.2.1
- 56Functions Under LoadCore architecture · Architecture World 5 · Skill 1.1.9
Articles on DEA-C01
DEA-C01 · Data engineering
Why Athena reads 1.2 TB to answer a four-column query
Athena bills the bytes it scans. Text is read whole; Parquet and ORC read only the columns a query names. How format, compression and file size set the bill.
Ends at Shape the FilesFree5 min read · video coming
SCS-C03 · Security
Identity policy or bucket policy: which one has to allow?
In one account, either side can allow. Across accounts, both must. An account principal only delegates, and Block Public Access overrides a public policy.
Ends at Deny WinsFree6 min read · video coming
Guide sections the path practises
| Section | As the guide words it | Stages |
|---|---|---|
| Skill 1.1.1 | Read data from streaming sources (for example, Amazon Kinesis, Amazon Managed Streaming for Apache Kafka [Amazon MSK], Amazon DynamoDB Streams, AWS DMS, AWS Glue, Amazon Redshift) | 7 |
| Skill 1.1.2 | Read data from batch sources (for example, Amazon S3, AWS Glue, Amazon EMR, AWS DMS, Amazon Redshift, AWS Lambda, Amazon AppFlow) | 3 |
| Skill 1.1.3 | Implement appropriate configuration options for batch ingestion | 1 |
| Skill 1.1.4 | Consume data APIs | 1 |
| Skill 1.1.5 | Set up schedulers by using Amazon EventBridge, Apache Airflow, or time-based schedules for jobs and crawlers | 2 |
| Skill 1.1.6 | Set up event triggers (for example, Amazon S3 Event Notifications, EventBridge) | 1 |
| Skill 1.1.7 | Call a Lambda function from Kinesis | 2 |
| Skill 1.1.8 | Create allowlists for IP addresses to allow connections to data sources | 1 |
| Skill 1.1.9 | Implement throttling and overcoming rate limits (for example, DynamoDB, Amazon RDS, Kinesis) | 5 |
| Skill 1.1.10 | Manage fan-in and fan-out for streaming data distribution | 3 |
| Skill 1.1.11 | Describe replayability of data ingestion pipelines | 3 |
| Skill 1.1.12 | Define stateful and stateless data transactions | 1 |
| Skill 1.2.1 | Optimize container usage for performance needs (for example, Amazon EKS, Amazon ECS) | 2 |
| Skill 1.2.2 | Connect to different data sources (for example, Java Database Connectivity [JDBC], Open Database Connectivity [ODBC]) | 1 |
| Skill 1.2.3 | Integrate data from multiple sources | 2 |
| Skill 1.2.4 | Optimize costs while processing data | 4 |
| Skill 1.2.5 | Implement data transformation services based on requirements (for example, Amazon EMR, AWS Glue, Lambda, Amazon Redshift) | 1 |
| Skill 1.2.6 | Transform data between formats (for example, from .csv to Apache Parquet) | 3 |
| Skill 1.2.7 | Troubleshoot and debug common transformation failures and performance issues | 3 |
| Skill 1.2.8 | Create data APIs to make data available to other systems by using AWS services | 1 |
| Skill 1.2.9 | Define volume, velocity, and variety of data (for example, structured data, unstructured data) | 1 |
| Skill 1.2.10 | Integrate large language models (LLMs) for data processing | 1 |
| Skill 1.3.1 | Use orchestration services to build workflows for data ETL pipelines (for example, Lambda, EventBridge, Amazon Managed Workflows for Apache Airflow [Amazon MWAA], AWS Step Functions, AWS Glue workflows) | 2 |
| Skill 1.3.2 | Build data pipelines for performance, availability, scalability, resiliency, and fault tolerance | 3 |
| Skill 1.3.3 | Implement and maintain serverless workflows | 1 |
| Skill 1.3.4 | Use notification services to send alerts (for example, Amazon SNS, Amazon SQS) | 2 |
| Skill 1.4.2 | Configure Lambda functions to meet concurrency and performance needs | 3 |
| Skill 2.1.1 | Implement the appropriate storage services for specific cost and performance requirements (for example, Amazon Redshift, Amazon EMR, AWS Lake Formation, Amazon RDS, Amazon DynamoDB, Amazon Kinesis Data Streams, Amazon Managed Streaming for Apache Kafka [Amazon MSK]) | 3 |
| Skill 2.1.2 | Configure the appropriate storage services for specific access patterns and requirements (for example, Amazon Redshift, Amazon EMR, Lake Formation, Amazon RDS, DynamoDB) | 1 |
| Skill 2.1.3 | Apply storage services to appropriate use cases (for example, using indexing algorithms like Hierarchical Navigable Small Worlds [HNSW] with Amazon Aurora PostgreSQL and using Amazon MemoryDB for fast key/value pair access) | 2 |
| Skill 2.1.4 | Integrate migration tools into data processing systems (for example, AWS Transfer Family) | 1 |
| Skill 2.1.5 | Implement data migration or remote access methods (for example, Amazon Redshift federated queries, Amazon Redshift materialized views, Amazon Redshift Spectrum) | 1 |
| Skill 2.1.6 | Manage locks to prevent access to data (for example, Amazon Redshift, Amazon RDS) | 2 |
| Skill 2.1.7 | Manage open table formats (for example Apache Iceberg) | 1 |
| Skill 2.1.8 | Describe vector index types (for example, HNSW, IVF) | 1 |
| Skill 2.2.1 | Use data catalogs to consume data from the data's source | 1 |
| Skill 2.2.2 | Build and reference a technical data catalog (for example, AWS Glue Data Catalog, Apache Hive metastore) | 1 |
| Skill 2.2.3 | Discover schemas and use AWS Glue crawlers to populate data catalogs | 2 |
| Skill 2.2.4 | Synchronize partitions with a data catalog | 1 |
| Skill 2.2.5 | Create new source or target connections for cataloging (for example, AWS Glue) | 1 |
| Skill 2.2.6 | Create and manage business data catalogs (for example, Amazon SageMaker Catalog) | 1 |
| Skill 2.3.1 | Perform load and unload operations to move data between Amazon S3 and Amazon Redshift | 1 |
| Skill 2.3.2 | Manage S3 Lifecycle policies to change the storage tier of S3 data | 1 |
| Skill 2.3.3 | Expire data when it reaches a specific age by using S3 Lifecycle policies | 1 |
| Skill 2.3.4 | Manage S3 versioning and DynamoDB TTL | 1 |
| Skill 2.3.5 | Delete data to meet business and legal requirements | 2 |
| Skill 2.3.6 | Protect data with appropriate resiliency and availability | 2 |
| Skill 2.4.1 | Design schemas for Amazon Redshift, DynamoDB, and Lake Formation | 1 |
| Skill 2.4.2 | Address changes to the characteristics of data | 2 |
| Skill 2.4.3 | Perform schema conversion (for example, by using AWS SCT and AWS DMS Schema Conversion) | 1 |
| Skill 2.4.4 | Establish data lineage by using AWS tools (for example, Amazon SageMaker ML Lineage Tracking and Amazon SageMaker Catalog) | 2 |
| Skill 2.4.5 | Describe best practices for indexing, partitioning strategies, compression, and other data optimization techniques | 4 |
| Skill 2.4.6 | Describe vectorization concepts (for example, Amazon Bedrock knowledge base) | 1 |
| Skill 3.1.1 | Orchestrate data pipelines (for example, Amazon MWAA, AWS Step Functions) | 1 |
| Skill 3.1.2 | Troubleshoot Amazon managed workflows | 1 |
| Skill 3.1.4 | Use the features of AWS services to process data (for example, Amazon EMR, Amazon Redshift, AWS Glue) | 1 |
| Skill 3.1.6 | Prepare data for transformation (for example, AWS Glue DataBrew and Amazon SageMaker Unified Studio) | 1 |
| Skill 3.1.7 | Query data (for example, Amazon Athena) | 4 |
| Skill 3.1.8 | Use AWS Lambda to automate data processing | 3 |
| Skill 3.1.9 | Manage events and schedulers (for example, Amazon EventBridge) | 2 |
| Skill 3.2.1 | Visualize data by using AWS services and tools (for example, DataBrew, Amazon QuickSight) | 1 |
| Skill 3.2.2 | Verify and clean data (for example, Lambda, Athena, QuickSight, Jupyter Notebooks, Amazon SageMaker Data Wrangler) | 1 |
| Skill 3.2.3 | Use SQL in Amazon Redshift and Athena to query data or to create views | 2 |
| Skill 3.2.4 | Use Athena notebooks that use Apache Spark to explore data | 1 |
| Skill 3.2.5 | Describe tradeoffs between provisioned services and serverless services | 3 |
| Skill 3.2.6 | Define data aggregation, rolling average, grouping, and pivoting | 1 |
| Skill 3.3.1 | Extract logs for audits | 1 |
| Skill 3.3.2 | Deploy logging and monitoring solutions to facilitate auditing and traceability | 1 |
| Skill 3.3.3 | Use notifications during monitoring to send alerts | 1 |
| Skill 3.3.4 | Troubleshoot performance issues | 4 |
| Skill 3.3.5 | Use AWS CloudTrail to track API calls | 1 |
| Skill 3.3.6 | Troubleshoot and maintain pipelines (for example, AWS Glue, Amazon EMR) | 2 |
| Skill 3.3.7 | Use Amazon CloudWatch Logs to log application data (with a focus on configuration and automation) | 1 |
| Skill 3.3.8 | Analyze logs with AWS services (for example, Athena, Amazon EMR, Amazon OpenSearch Service, CloudWatch Logs Insights, big data application logs) | 2 |
| Skill 3.4.1 | Run data quality checks while processing the data (for example, checking for empty fields) | 2 |
| Skill 3.4.2 | Define data quality rules (for example, DataBrew) | 1 |
| Skill 3.4.3 | Investigate data consistency (for example, DataBrew) | 1 |
| Skill 3.4.4 | Describe data sampling techniques | 1 |
| Skill 3.4.5 | Implement data skew mechanisms | 1 |
| Skill 4.1.1 | Update VPC security groups | 2 |
| Skill 4.1.2 | Create and update IAM groups, roles, endpoints, and services | 1 |
| Skill 4.1.3 | Create and rotate credentials for password management (for example, AWS Secrets Manager) | 2 |
| Skill 4.1.4 | Set up IAM roles for access (for example, AWS Lambda, Amazon API Gateway, AWS CLI, AWS CloudFormation) | 1 |
| Skill 4.1.5 | Apply IAM policies to roles, endpoints, and services (for example, S3 Access Points, AWS PrivateLink) | 2 |
| Skill 4.1.7 | Use domain, domain units, and projects for SageMaker Unified Studio | 1 |
| Skill 4.2.1 | Create custom IAM policies when a managed policy does not meet the needs | 2 |
| Skill 4.2.2 | Store application and database credentials (for example, Secrets Manager, AWS Systems Manager Parameter Store) | 2 |
| Skill 4.2.3 | Provide database users, groups, and roles access and authority in a database (for example, for Amazon Redshift) | 1 |
| Skill 4.2.4 | Manage permissions through AWS Lake Formation (for Amazon Redshift, Amazon EMR, Amazon Athena, and Amazon S3) | 3 |
| Skill 4.2.5 | Apply authorization methods that address business needs (role-based, tag-based, and attribute-based) | 3 |
| Skill 4.2.6 | Construct custom policies that meet the principle of least privilege | 3 |
| Skill 4.3.1 | Apply data masking and anonymization according to compliance laws or company policies | 2 |
| Skill 4.3.2 | Use encryption keys to encrypt or decrypt data (for example, AWS KMS) | 3 |
| Skill 4.3.3 | Configure encryption across AWS account boundaries | 4 |
| Skill 4.3.4 | Enable encryption in transit or before transit for data | 2 |
| Skill 4.4.1 | Use AWS CloudTrail to track API calls | 3 |
| Skill 4.4.2 | Use Amazon CloudWatch Logs to store application logs | 1 |
| Skill 4.4.3 | Use AWS CloudTrail Lake for centralized logging queries | 4 |
| Skill 4.4.4 | Analyze logs by using AWS services (for example, Athena, CloudWatch Logs Insights, Amazon OpenSearch Service) | 1 |
| Skill 4.4.5 | Integrate various AWS services to perform logging (for example, Amazon EMR in cases of large volumes of log data) | 1 |
| Skill 4.5.1 | Grant permissions for data sharing (for example, data sharing for Amazon Redshift) | 2 |
| Skill 4.5.2 | Implement PII identification (for example, Amazon Macie with Lake Formation) | 1 |
| Skill 4.5.3 | Implement data privacy strategies to prevent backups or replications of data to disallowed AWS Regions | 2 |
| Skill 4.5.4 | Viewing configuration changes that have occurred in an account (for example, AWS Config) | 2 |
| Skill 4.5.5 | Maintain data sovereignty | 1 |
| Skill 4.5.6 | Manage data access through Amazon SageMaker Catalog projects | 1 |
| Skill 4.5.7 | Describe governance data framework and data sharing patterns | 2 |
A domain and a path are separate products. What you own is the union of what you bought, so a stage you already own is never sold to you twice: the game offers the cheapest way to unlock the stage in front of you. Path $15, exam $150.