PLAN-101 Career OS
All Levels Estimated: 12 Weeks 12 Topics

Data Engineer Roadmap

Comprehensive 4-phase roadmap for Data Engineer covering 12 key topics with practical deliverables.

Curriculum Syllabus & Milestones

Phase 1: Foundations

Focus area covering 3 core topics in Foundations.

  1. Python & SQL

    Sharpen Python scripting and advanced SQL (window functions, CTEs).

    • Study primary concepts for: Python & SQL
    • Actionable guide: Sharpen Python scripting and advanced SQL (window functions, CTEs).
    • Build or test a practical mini-project applying Python & SQL.
  2. Data Sources & Formats

    Work with CSV, JSON, Parquet and understand structured vs unstructured data.

    • Study primary concepts for: Data Sources & Formats
    • Actionable guide: Work with CSV, JSON, Parquet and understand structured vs unstructured data.
    • Build or test a practical mini-project applying Data Sources & Formats.
  3. Linux & Git

    Practice shell scripting and Git workflows used in data pipelines.

    • Study primary concepts for: Linux & Git
    • Actionable guide: Practice shell scripting and Git workflows used in data pipelines.
    • Build or test a practical mini-project applying Linux & Git.

Phase 2: Core Skills

Focus area covering 3 core topics in Core Skills.

  1. Build a Data Pipeline

    Write an ETL script that extracts, transforms and loads data into a database.

    • Study primary concepts for: Build a Data Pipeline
    • Actionable guide: Write an ETL script that extracts, transforms and loads data into a database.
    • Build or test a practical mini-project applying Build a Data Pipeline.
  2. Orchestration with Airflow

    Convert your script into a scheduled Airflow DAG.

    • Study primary concepts for: Orchestration with Airflow
    • Actionable guide: Convert your script into a scheduled Airflow DAG.
    • Build or test a practical mini-project applying Orchestration with Airflow.
  3. Data Warehousing

    Load and model data in a warehouse (BigQuery/Snowflake) using star schema.

    • Study primary concepts for: Data Warehousing
    • Actionable guide: Load and model data in a warehouse (BigQuery/Snowflake) using star schema.
    • Build or test a practical mini-project applying Data Warehousing.

Phase 3: Applied Practice

Focus area covering 3 core topics in Applied Practice.

  1. Streaming Basics

    Explore Kafka fundamentals with a simple producer/consumer example.

    • Study primary concepts for: Streaming Basics
    • Actionable guide: Explore Kafka fundamentals with a simple producer/consumer example.
    • Build or test a practical mini-project applying Streaming Basics.
  2. dbt for Transformations

    Model raw data into clean tables using dbt with tests.

    • Study primary concepts for: dbt for Transformations
    • Actionable guide: Model raw data into clean tables using dbt with tests.
    • Build or test a practical mini-project applying dbt for Transformations.
  3. Data Quality & Testing

    Add data validation checks (Great Expectations or dbt tests) to your pipeline.

    • Study primary concepts for: Data Quality & Testing
    • Actionable guide: Add data validation checks (Great Expectations or dbt tests) to your pipeline.
    • Build or test a practical mini-project applying Data Quality & Testing.

Phase 4: Career

Focus area covering 3 core topics in Career.

  1. Cloud Data Services

    Use a managed cloud data pipeline service (Glue/Dataflow) for one small job.

    • Study primary concepts for: Cloud Data Services
    • Actionable guide: Use a managed cloud data pipeline service (Glue/Dataflow) for one small job.
    • Build or test a practical mini-project applying Cloud Data Services.
  2. Infrastructure as Code

    Provision your pipeline infra (buckets, warehouse) with Terraform.

    • Study primary concepts for: Infrastructure as Code
    • Actionable guide: Provision your pipeline infra (buckets, warehouse) with Terraform.
    • Build or test a practical mini-project applying Infrastructure as Code.
  3. Portfolio Pipeline

    Publish an end-to-end pipeline project with documentation and architecture diagram.

    • Study primary concepts for: Portfolio Pipeline
    • Actionable guide: Publish an end-to-end pipeline project with documentation and architecture diagram.
    • Build or test a practical mini-project applying Portfolio Pipeline.

Related Career Paths & Cross-Skill Roadmaps

Expand your engineering versatility by exploring interconnected role roadmaps:

3-Year CSE Roadmap → Frontend Developer → Backend Developer → Full-Stack Developer →

Track Your Data Engineer Roadmap Progress

Sign in with Google to check off completed topics, save code notes, and monitor your placement readiness score.

Start Tracking Roadmap