Zenler Player
Your course is loading. Hang tight.
Spark DataFrame Mastery
Back to curriculum
0% Complete
0% Complete
1 30 Day Pyspark Coding Challenge - Intro
2 Recommendations & Your Commitment ---
3 Set up your Account for Pyspark Challenge
4 Teardown When Done
Day 1 Spark Fundamentals: SparkSession & RDDs
Day 1 - 1 - Spark Application & Spark Session VIMP ---
Day 1 - 2 - Easiest way and the most inexpensive way to learn PySpark Coding ---
Day 1 - 3 - Dataframes and RDDs again ---
Day 1 - 4 - RDDs ---
Day 2 DataFrames vs RDDs
Day 2 - RDDs and Dataframes again ... ---
Day 3 Schema & DataFrame Reader
Day 3 - 1 - Reader API and different options ---
Day 3 - 2 - InferSchema vs Creating and supplying your own Schema ---
Day 3 - 3 - Read Mode Permissive and finding corrupted record ---
Day 3 - 4 - Read Mode DROPMALFORMED and FAILFAST ---
Day 4 DataFrame Writer, Compression & Partitioning
Day 4 - 1 - Writer API Modes
Day 4 - 2 - Writer API Modes
Day 4 - 3 - Parquet
Day 4 - 4 - Compression Techniques - Important
Day 4 - 5 - Creating and Reading from Partitions - Important
Day 4 - 6 - Real World Link
Day 5 - Selecting Columns
Day 5 Selecting Columns from Dataframe
Day 6 withColumn, withColumnRenamed & Dropping Columns
Day 6 withColumn, withColumnRenamed & Dropping Columns
Day 7 Filtering, Literal & Casting
Day 7 Filtering, Literal & Casting
Day 8 Creating DataFrames, Unique Records & Dropping Duplicates
Day 8 Creating DataFrames, Unique Records & Dropping Duplicates
SQL equivalent in Pyspark
How Interviewers test your PySpark Skills?
Day 9 when / otherwise — Conditional Logic
Day 10 Handling Nulls
Day 11 String Functions
Day 12 Date Functions
Day 13 Joins — inner, left, right, full outer
Day 14 Semi, Anti & Cross Joins — union & unionByName
Day 15 GroupBy & Aggregations
Day 16 Window Functions: row_number, rank, dense_rank
Day 17 Window Functions: lag, lead & Running Totals
Day 18 UDFs — User Defined Functions
Day 18 - 1- Python UDFs - Machine Code, Compilation & Bytecode
Day 18 - 2- Python UDFs - JVM, Spark and Garbage Collection
Day 18 - 3- Python UDFs - Py4J
Day 18 - 4- Python UDFs - How to create one and use them
Day 18 - 5- Python UDFs - How to create one and use them
Day 19 JSON — Reading, explode & Nested Data
Day 19 JSON — Reading, explode & Nested Data
Day 20 - 1 - Spark SQL
Day 20 - 2 - Spark SQL
Day 20 Spark SQL
Day 21 Mini Project
Day 21 Mini Project
1 - What's Coming Next? Are you committed?
Day 22 Practice Set 1 — Filtering & Aggregations
Day 23 Practice Set 2 — Joins
Day 24 Practice Set 3 — Window Functions
Day 25 Practice Set 4 — String & Date Functions
Day 26 Practice Set 5 — Nulls & Data Quality
Day 27 Practice Set 6 — Mixed Problems
Day 28 Practice Set 7 — Hard Multi-Step Pipelines
Day 29 Practice Set 8 — Company-Style Problems
Day 30 Full Mock Coding Interview Round
2 - You are awesome! DEH Hall of Fame is waiting for you!
Start Here
1 30 Day Pyspark Coding Challenge - Intro
Preview
2 Recommendations & Your Commitment ---
3 Set up your Account for Pyspark Challenge
4 Teardown When Done
Part 1: Days 1–8: The Commitment Phase
Day 1 Spark Fundamentals: SparkSession & RDDs
Day 1 - 1 - Spark Application & Spark Session VIMP ---
Day 1 - 2 - Easiest way and the most inexpensive way to learn PySpark Coding ---
Day 1 - 3 - Dataframes and RDDs again ---
Day 1 - 4 - RDDs ---
Day 2 DataFrames vs RDDs
Day 2 - RDDs and Dataframes again ... ---
Day 3 Schema & DataFrame Reader
Day 3 - 1 - Reader API and different options ---
Day 3 - 2 - InferSchema vs Creating and supplying your own Schema ---
Day 3 - 3 - Read Mode Permissive and finding corrupted record ---
Day 3 - 4 - Read Mode DROPMALFORMED and FAILFAST ---
Day 4 DataFrame Writer, Compression & Partitioning
Day 4 - 1 - Writer API Modes
Day 4 - 2 - Writer API Modes
Day 4 - 3 - Parquet
Day 4 - 4 - Compression Techniques - Important
Day 4 - 5 - Creating and Reading from Partitions - Important
Day 4 - 6 - Real World Link
Day 5 - Selecting Columns
Day 5 Selecting Columns from Dataframe
Day 6 withColumn, withColumnRenamed & Dropping Columns
Day 6 withColumn, withColumnRenamed & Dropping Columns
Day 7 Filtering, Literal & Casting
Day 7 Filtering, Literal & Casting
Day 8 Creating DataFrames, Unique Records & Dropping Duplicates
Day 8 Creating DataFrames, Unique Records & Dropping Duplicates
Part 2: Days 9–17: The Builder Phase
SQL equivalent in Pyspark
How Interviewers test your PySpark Skills?
Day 9 when / otherwise — Conditional Logic
Day 10 Handling Nulls
Day 11 String Functions
Day 12 Date Functions
Day 13 Joins — inner, left, right, full outer
Day 14 Semi, Anti & Cross Joins — union & unionByName
Day 15 GroupBy & Aggregations
Day 16 Window Functions: row_number, rank, dense_rank
Day 17 Window Functions: lag, lead & Running Totals
Part 3 Days 18–21 — The Mastery Phase
Day 18 UDFs — User Defined Functions
Day 18 - 1- Python UDFs - Machine Code, Compilation & Bytecode
Day 18 - 2- Python UDFs - JVM, Spark and Garbage Collection
Day 18 - 3- Python UDFs - Py4J
Day 18 - 4- Python UDFs - How to create one and use them
Day 18 - 5- Python UDFs - How to create one and use them
Day 19 JSON — Reading, explode & Nested Data
Day 19 JSON — Reading, explode & Nested Data
Day 20 - 1 - Spark SQL
Day 20 - 2 - Spark SQL
Day 20 Spark SQL
Day 21 Mini Project
Day 21 Mini Project
Part 4 Days 22–30 — The Interview Cracking Phase
1 - What's Coming Next? Are you committed?
Day 22 Practice Set 1 — Filtering & Aggregations
Day 23 Practice Set 2 — Joins
Day 24 Practice Set 3 — Window Functions
Day 25 Practice Set 4 — String & Date Functions
Day 26 Practice Set 5 — Nulls & Data Quality
Day 27 Practice Set 6 — Mixed Problems
Day 28 Practice Set 7 — Hard Multi-Step Pipelines
Day 29 Practice Set 8 — Company-Style Problems
Day 30 Full Mock Coding Interview Round
2 - You are awesome! DEH Hall of Fame is waiting for you!
×
This is an unpublished lesson. This lesson will not be shown for students unless you set it as Public.
Back to Dashboard
No contents are available in this lesson!
No lessons available !
Back to Dashboard
Lesson contents locked
Enroll to unlock this lesson.
Enroll to unlock
Next Lesson