Dewei Zhai
Menu

/cv

I turn slow, costly, and fragmented data delivery into reliable production systems.

10 years working across 500TBโ€“1PB data platforms on AWS, Azure, GCP, and Alibaba Cloudโ€”from building a startup foundation alone to evolving platforms used by 100+ person data organisations.

My advantage is finding the mismatch between the business loop, workload lifecycle, technical architecture, and team boundaries. I turn that diagnosis into a simpler delivery system with explicit governance and measurable outcomes.

The environments I have worked in

From building a foundation alone to improving a live platform across a large organisation.

Startup ยท 0 โ†’ 1 ยท Enyquant

A model-ready national dataset in 60 days

A new energy company needed five years of operational and market data for one complete country before its SMEs could train market-price forecasting models.

There was no existing platform team; I delivered the platform end to end, from architecture and code through deployment and daily operations.

  • โ€” Delivered the complete five-year DuckDB dataset within 60 days from the start of platform construction.
  • โ€” Defined point-in-time visibility semantics across nearly all datasets and model-training flows.
  • โ€” Downstream SMEs used the released data to train working models; further details remain confidential.

Large data organisation ยท evolve in place ยท VodafoneZiggo

Move hundreds of terabytes and cut feedback from hours to minutes

A 100+ person Data Tribe was moving hundreds of terabytes from Oracle to Snowflake while its shared 1PB+ core data platform remained live.

As a cloud administrator and data engineer, I supported the migration through AWS DMS Full Load + CDC and built the workflow automation and validation path.

  • โ€” My remit covered DMS tasks, mappings, monitoring, recovery, reconciliation, and production support; the wider migration was a cross-team effort.
  • โ€” Embedded PowerCenter conversion and early checks into GitLab CI/CD for at least 40 developers.
  • โ€” Feedback moved from hours or days to minutes; conservatively at least 40 hours of manual work were removed each week.

Larger organisations ยท small core teams ยท PVH ยท FedEx

Turn a 4โ€“6 week dashboard path into 1โ€“2 hours

At PVH, analyst notebook logic crossed separate analyst and engineering queues before it could become a governed production dataset.

In a roughly ten-person team without a Data Architect, I designed the platform path and acted as its de facto architecture lead.

  • โ€” A YAML contract, static checks, experiments layer, data quality, and a DAG factory replaced the two-queue rewrite path.
  • โ€” End-to-end delivery fell from 4โ€“6 weeks to about 1โ€“2 hours across a platform serving 60+ dashboards; review and deployment took about 10โ€“20 minutes once SQL was complete.

Track record

  1. Feb 2026 โ€” now

    Data Platform Owner & Platform ArchitectEnyquant

    Enyquant is an energy-intelligence startup building forecasting and decision-support systems for electricity markets. As Data Platform Owner, I built its cloud and data foundation from zeroโ€”from architecture and implementation to deployment and daily operations. I delivered five years of national electricity operations and market data within 60 days, defined point-in-time visibility across nearly all datasets and model-training flows, and re-platformed daily work from Databricks + ADF to a single-VM DuckDB, Polars and DuckLake architecture, reducing comparable monthly cost by about 95%.

  2. Feb 2019 โ€” Jan 2026

    OwnerDewei Consulting B.V.

    My independent consulting company in the Netherlands, used to help organisations build and modernise data platforms, productionise complex workloads and improve delivery across engineering and business teams. The work covered architecture, hands-on implementation, cloud infrastructure, pipelines, CI/CD, governance and production operations for clients including PVH, VodafoneZiggo and FedEx. Business activity paused in 2026 as I shifted my main focus to independent AI product exploration.

  3. Nov 2023 โ€” Oct 2025

    Lead Data EngineerPVH Corp ยท 2nd engagement

    Returned to a roughly ten-person core team supporting a 500+TB AWS data platform. With no dedicated Data Architect, I acted as the de facto architecture lead. I redesigned a two-team dashboard path with YAML contracts, static validation, an experimentation layer, data-quality controls and a DAG factory; a representative end-to-end cycle fell from 4โ€“6 weeks to about 1โ€“2 hours and supported 60+ dashboards. Once SQL was complete, review and deployment took about 10โ€“20 minutes. I also rebuilt an approved PII re-identification flow as event-driven serverless: typical latency fell from about two hours to 5โ€“10 minutes and related AWS cost by about 90%. I participated in Azure Databricks reviews and migration support without owning the overall migration.

  4. Sep 2022 โ€” Jan 2024

    Senior Data EngineerVodafoneZiggo

    Worked within a Data Tribe of 100+ people on a live 1PB+ core data platform, contributing inside established architecture governance. I evolved an existing Scala/Spark ETL framework; built and operated AWS DMS Full Load + CDC tasks for a hundreds-of-terabytes Oracle-to-Snowflake migration; and embedded PowerCenter conversion and early validation in GitLab CI/CD. Feedback for at least 40 developers moved from hours or days to minutes, conservatively removing 40+ hours of manual work each week. I also built JupyterHub + MLflow services for 4โ€“5 data scientists.

  5. Dec 2020 โ€” Aug 2022

    Senior Data EngineerPVH Corp ยท 1st engagement

    Joined as the original Hadoop-to-AWS migration neared completion, focusing on production-platform evolution and reusable engineering patterns. I designed Adobe, Salesforce and SAP integrations, developed idempotent configuration-driven ETL, improved observability, data quality and cross-time-zone scheduling, and contributed to continued operation of the 500+TB AWS platform.

  6. Aug 2018 โ€” Nov 2020

    Data EngineerFedEx (formerly TNT Digital International)

    Worked in a 10โ€“15 person DE/DS team within the roughly 200-person former TNT Digital organisation. I managed AWS and GCP infrastructure with Terraform, built and operated Spark/EMR ETL and integrations, developed and ran Spark workloads on an existing GKE cluster, and supported CI/CD, production operations and JupyterHub. A Spark job covering at least 175 business cases previously failed with OOM after three hours; the reworked version completed the full workload in under five minutes.

  7. Feb 2018 โ€” Jul 2018

    Data EngineerABN AMRO

    Contributed to DIAL, an enterprise platform consolidating fragmented departmental ETL onto a shared environment. I migrated workloads from Hive to PySpark, built Python/Spark/Hive pipelines, and replaced a recurring manual Excel workflow taking about three days per week with a Python process completing in roughly three minutes.

  8. Nov 2016 โ€” Jan 2018

    Big Data ConsultantKPN

    Delivered and operated Hortonworks HDP clusters on bare-metal infrastructure for enterprise customers. I installed and configured clusters, developed Hive/Spark ETL, automated infrastructure and delivery with Python, Ansible and Jenkins, and built an end-to-end Robot Framework health-validation suite that replaced repetitive post-installation checks and saved the team more than eight hours per week.

  9. Oct 2012 โ€” Nov 2016

    Network Quality EngineerKPN

    Owned technical validation and production acceptance for telecom network solutions: reviewed high- and low-level designs, coordinated implementation and testing in acceptance environments, identified technical and operational risks before handover, and supported controlled transition into production. Used Python and PostgreSQL for engineering analysis and automation.

  10. Jun 2009 โ€” Oct 2012

    Software Test EngineerHuawei

    Worked on integration, verification and customer delivery of Huawei HLR/HSS core-network database systems. As test and delivery lead for a major KPN Netherlands cutover involving a system with capacity for 16M subscriber lines, I ran about seven months of intensive testing, identified hundreds of defects, and helped design and execute staged migration and rollback procedures. The programme migrated 12M subscribers with zero production incidents; I also supported pre-sales and post-sales delivery.

Selected work

A few outcomes that represent how I work. Related articles document the design and delivery decisions behind them.

Tech stack

Cloud
AWS (deepest), Azure, Alibaba Cloud, GCP
Lakehouse
Databricks, Snowflake, Unity Catalog, Iceberg, DuckLake, DuckDB
ETL
Spark, Polars, dbt, Glue, Kafka, AWS DMS (CDC)
Languages
Python, SQL, TypeScript, Scala, Shell, Solidity, Cython
Orchestration
Airflow, ADF, Step Functions, Oozie
IaC
Terraform, AWS CDK, CDKTF, Ansible
CICD
GitHub Actions, GitLab CI/CD, Jenkins
Quality
PyDeequ, Great Expectations

Publications

Certifications

  • โ€” AWS Certified Solutions Architect โ€” Associate
  • โ€” Databricks Certified Associate Developer for Apache Spark
  • โ€” Databricks Certified Data Engineer Associate
  • โ€” Certified Associate in Python Programming

Education

MSc, Communication & Information Systems โ€” Xidian University, China (Telecommunication Engineering: #4 globally in ShanghaiRanking's 2025 Global Ranking of Academic Subjects). IDW evaluation: equivalent to MSc Computing Science (NL).