Stefan D. Email & Phone Number

Data Engineer with 3 years of experience specialising in ETL orchestration and data system integrations. Proficient in Python, Snowflake, and distributed systems, with a proven track record of enhancing data analysis and business intelligence capabilities. Adept at leveraging data modelling techniques and cloud technologies to drive efficiency, accuracy, and scalability in complex data environments.

• 

  Data Engineer

  Ppl

  Dec 2023 - Present - London, England, United Kingdom

  Developed a new Snowflake-based analytical data platform, significantly enhancing data analysis capabilities and supporting the distribution of over £280 million in royalties to 140,000 artists and labels.Integrated multiple source systems into the data warehouse, utilising dbt for the management of approximately 1,000 models and implementing Data Vault 2.0 for sophisticated data modelling.Built and optimised AWS Glue (PySpark) jobs, leading to an 80% improvement in runtime and cost efficiency.Developed a custom PySpark bookmarking functionality to facilitate incremental loading of S3 files using AWS Glue, improving flexibility and transparency and making associated jobs more fault tolerant, ensuring reliable data ingestion.SME in using Airflow(AWS MWAA), creating and maintaining 8 DAGs to streamline process orchestration.Implemented a bespoke alerting system that interconnected various processes and facilitated custom Slack notifications, drastically cutting down troubleshooting time and improving stakeholder visibility.