-
0 Get Ready for the Roller Coaster Ride
1 Lesson-
PreviewComment - Your effort can change your career trajectory!
-
-
1 Create AWS Account & Set Up AWS CLI
11 Lessons-
StartAWS Set up
-
Start1 Intro - AWS Setup
-
Start2 Create AWS Account
-
Start3 Login to AWS using Root User and go to IAM
-
Start4 Create Admin User with Console and Programmatic Access
-
Start5 Download and Install AWS CLI
-
Start6 Create Access Key for AWS CLI Access
-
Start7 Configure AWS CLI on your system
-
Start8 VIMP - Set up three AWS Budgets
-
Start8.5 Set Default Region to us-east-1
-
Start9 Comment needed - Outtro - AWS Set Up
-
-
2 Set-Up For Data Analysis
11 Lessons-
StartSet up for Data Analysis
-
Start1a. Intro - Set up for Analysis
-
Start1b. Datalake Setup
-
Start2. Download Data from NYC TLC
-
Start3. Move the downloaded file to your project folder
-
Start4. Create S3 bucket in Datalake & upload data to the datalake S3 bucket
-
Start5. Comment Needed - Hands On Starts
-
Start6. Create Glue Catalog Database
-
Start7. Create a crawler to crawl the data and run the crawler
-
Start8. View the Parquet Data in Athena
-
Start9. Comment Needed Outro
-
-
3 Build an Ingestion ( Extract of ETL ) job
6 Lessons-
Start1. Comment Below - Mandatory detour to Course RADE™ Agentic Data Engineering with Amazon Q
-
Start2. Before the Hands on Continues - VIMP!
-
Start3. Create the Glue Ingestion Job
-
Start4. Run Glue Crawler again to update the Table Metadata with partitions
-
Start5. Create Crawler for Zone Table
-
Start6. Comment Below - Next Step is Very Important
-
-
4 Understanding and Gathering Requirements
6 Lessons-
StartBefore the Development Begins
-
Start:- 1. Intro - Understanding and Gathering Requirements
-
Start2. :- Comment -Capstone Requirements and Guidance on completing it - 3 options
-
Start3. Analysis by Data Analysts, Meeting with them and Proposing a 3 layer structure
-
Start4. Understanding Requirements using the Mapping Sheet
-
Start5. Comment Decide on Technical Requirements & gear up for development
-
-
5 Build the data transformation job - Raw to Curated
5 Lessons -
6 Unit Testing, analysis, and further development
13 Lessons-
Start0 Comment - What will you learn in this section
-
Start1 Time to unit test before we hand over the data to Data analysts
-
Start2 Unit test the curated layer and fill the unit test document and then hand over to the data anaysts
-
Start3 Analyse the Curated to Aggregated Mapping Sheet
-
Start4 Create Glue Notebook , load the curated data and enrich with Zone information
-
Start5 Create the aggregated tables and load the aggregated data into S3
-
Start6 Get the script from the notebook and stop the notebook
-
Start7 Get the script vetted by AI
-
Start8 Create and Run the Glue ETL job to load from curated to Aggregated
-
Start9 What-s the next step
-
Start10 Create aggregated tables with the help of Crawler
-
Start11 Unit test the aggregated data, provide access to Data analysts and let them to know
-
Start12 Comment - What have you learnt so far
-
-
7 Very Important Section - Historical & Incremental Loading
7 Lessons-
Start1 Historical Vs Monthly Cadence
-
Start2 Comment - Code for both historical and current - run the job
-
Start3 Why do we even need to think of incremental processing of Data
-
Start4 Extremely Important from Interview Perspective - Enable Job bookmarks for incremental processing
-
Start4.1 Glue Dynamic Frame vs Spark Dataframe
-
Start5 Comment Incremental Processing from Curated to Aggregated
-
Start6 Solution to previous ask
-
Fill the waitlist form to enroll into this course
AWS Data Engineering Project Labs
Build a real-world AWS data engineering project through guided, production-style labs.
Course Summary
RADE: AWS Data Engineering Project Labs is a fully hands-on course designed to help you learn AWS data engineering by building real systems.
This course is project and lab driven — you’ll spend your time provisioning AWS services, wiring them together, and running end-to-end data pipelines, just like in real-world data engineering roles.
There are no deep theoretical explanations of AWS services in this course.
Instead, each lab focuses on what to build, how to build it, and why it matters in real production scenarios.
If you ever feel gaps in understanding how a service works internally or why certain architectural choices exist, those topics are covered separately in RADE: AWS Data Engineering Deep Dive.
By the end of this course, you will have:
-
Built an end-to-end AWS data engineering pipeline
-
Gained confidence working hands-on with core AWS data engineering services
-
Developed practical experience you can confidently discuss in interviews and on your resume
This course is part of the RADE (Real-World AWS Data Engineering) Program and is included in the Silver Membership.
Course Curriculum
