The challenge behind the platform.
A large Talend ETL estate needed more maintainable orchestration and reusable cloud engineering patterns.
My contribution
Led the migration of 1,000+ Talend jobs to an AWS-based Airflow framework. Currently supporting the next stage of platform modernisation toward Databricks.
Inside the system.
Moved legacy ETL workloads into an AWS-based Apache Airflow framework.
Established reusable pipeline patterns and engineering standards for production workloads.
Supported team capability through mentoring and code reviews.
Now supporting the evolution of orchestration and processing toward managed Databricks capabilities.
Talend · AWS · Airflow · Databricks · Python / PySpark
From design to delivery.
Combined ETL leadership, complex SQL development and warehouse modelling with cloud-framework delivery. The next platform phase is an ongoing contribution rather than a completed migration claim.
Completed migration of 1,000+ ETL jobs to AWS / Airflow. The subsequent Databricks initiative is in progress; no completion or performance claims are made for that phase.