Data Engineering: Building Reliable Pipelines
- Hands-on AI and data science techniques from working practitioners
- Structured program with clear milestones and real-world datasets
- Accessible remotely — join from anywhere in Ukraine
About this program
Where data actually comes from
Most data science courses assume clean, ready-to-use datasets. In practice, someone has to build and maintain the systems that produce them. This program covers that work directly.
You will design ETL workflows, schedule jobs with Apache Airflow, and store data in structured formats on cloud object storage. The focus is on reliability: what happens when a pipeline fails at 3 AM and no one is watching.
Technical stack
- SQL (PostgreSQL) for transformations and data modeling
- Apache Airflow for orchestration
- dbt for SQL-based transformation layers
- Google Cloud Storage and BigQuery for cloud data warehousing
Who this suits
Analysts who write SQL regularly and want to move into engineering. Also suitable for backend developers adding data infrastructure skills to their profile.
Workload and format
Eight weeks, with two live sessions per week and weekly infrastructure assignments. Participants need access to a machine capable of running Docker locally.
Program structure
-
Week 1 — Data Modeling Fundamentals
Relational schemas, normalization, star and snowflake schemas for analytics.
-
Week 2–3 — ETL Pipeline Design
Extraction patterns, incremental loads, idempotency, error handling strategies.
-
Week 4 — Apache Airflow
DAG structure, operators, task dependencies, monitoring and alerting.
-
Week 5 — dbt for Transformations
Models, tests, documentation, incremental materialization.
-
Week 6 — Cloud Data Warehousing
BigQuery architecture, partitioning, query optimization, cost control.
-
Week 7–8 — Capstone Pipeline Project
Build a complete pipeline from raw source to analytics-ready table, with documentation and monitoring.