D24
Data & AIIntermediate12 weeks
Data Engineering
Learn to design, build, and operate production-grade data pipelines — from ingestion and warehousing to distributed batch processing, streaming, and cloud lakehouse platforms.
What you'll learn
Building ELT/ETL pipelines with Python, SQL, and dbt
Batch processing at scale with Spark and workflow orchestration with Airflow
Streaming pipelines with Kafka and cloud data warehousing on Snowflake or BigQuery
Using AI-assisted tooling for schema inference, anomaly detection, and pipeline documentation
Course details
Duration
12 weeks
Best for
Developers and analysts who want to design and operate reliable, production-grade data pipelines
On completion
Verified certificate & placement assistance
Course Curriculum
A module-by-module breakdown of what you'll cover in Data Engineering.
01
Environment & Data Engineering Foundations
Weeks 1–2- Dev environment setup, Linux, and Git workflows
- Advanced SQL: window functions and CTEs
- API ingestion, pagination, and idempotent script design
- Working with Parquet, JSON, and CSV formats
02
Data Modeling & Warehousing
Weeks 3–4- Star and snowflake schema design
- Dimensional modeling with facts and dimensions
- Slowly changing dimensions
- Building and testing dbt models
03
ETL/ELT & Batch Processing with Spark
Weeks 5–6- Batch vs. streaming ETL patterns
- Incremental loads, CDC, and backfills
- PySpark DataFrame API and joins at scale
- Performance tuning: partitioning, shuffling, and caching
04
Orchestration & Streaming
Weeks 7–8- Airflow DAG design, dependencies, and retries
- Scheduling, backfilling, and failure alerting
- Kafka producers, consumers, and consumer groups
- Spark Structured Streaming and windowed aggregations
05
Cloud Platforms & Data Quality
Weeks 9–10- Snowflake, BigQuery, and Databricks lakehouse architecture
- Query optimization, partitioning, and cost control
- Data validation suites and freshness monitoring
- Anomaly detection and lineage tracking
06
AI-Assisted Data Engineering
Weeks 11–12- Schema inference from raw files with an LLM
- AI-generated transformation code and SQL review
- Auto-generated pipeline documentation
- Guardrails for reviewing AI-generated code
Tools & Technologies
Hands-on practice with the same tools used by working professionals.
Python
SQL
dbtdbt
ApaApache Spark
ApaApache Airflow
ApaApache Kafka
SnoSnowflake / BigQuery
GreGreat Expectations
Hands-on Projects
This is project-based learning — you'll build real, portfolio-ready work as you go, not just watch lectures.
E-commerce data warehouse with a dimensional star schema and a Type 2 slowly changing dimension, built in dbt
Real-time clickstream pipeline streaming windowed session aggregates into a lakehouse table with Kafka and Spark
Batch ELT pipeline with automated data-quality gates that halt on schema violations or null-rate spikes
CDC replication pipeline capturing change events from Postgres and syncing them to a cloud warehouse in near real time
Cost-optimized cloud warehouse migration with before/after query-latency and cost benchmarks
Every course includes a guided internship on top of your project work — so you graduate with real experience, not just a certificate.
Communication & Soft Skills
Built into every course, alongside the technical curriculum.
Communication Skills
Practice explaining technical work clearly to teammates, managers, and clients.
Teamwork & Collaboration
Work in small project teams using real workflows like stand-ups and code reviews.
Problem Solving
Build the habit of breaking down ambiguous problems into clear, solvable steps.
Time Management
Learn to plan, prioritize, and deliver project work against realistic deadlines.
Presentation Skills
Present your project work and results with confidence, as you would to a client.
Resume & Profile Building
Build a strong resume, LinkedIn profile, and project portfolio that recruiters notice.
Interview Preparation
Mock interviews and portfolio reviews to get you ready for real job applications.
Student Reviews
“The Spark and Airflow modules were exactly what I needed to move from writing scripts to building pipelines that actually run reliably in production.”
Arvind Menon
Data Engineer, Fintech Startup
“Building the dbt star schema project end-to-end made dimensional modeling click in a way no tutorial ever had. The streaming module was tough but worth it.”
Sneha Kulkarni
Analytics Engineer
