Led a 5-person team managing up to thirty data pipelines simultaneously, processing trillions of records per quarter. Owned and improved the company's ETL process, driving reductions in data intake and processing times by 50% through expanding team capabilities and technical solutions. Led hiring and team expansion, fostering professional growth through regular 1-on-1s and skill development. Automated pipeline tasks, enhanced error handling, and improved documentation, working towards a "No-Touch" ETL process. Regularly presented metrics to stakeholders, while ensuring data security compliance and managing client-related issues.

Collaborated with a small team to develop custom Hadoop MapReduce code for Apache Spark clusters, transforming Electronic Medical Records (EMR) and Real-World Data (RWD) into proprietary data models. Partnered closely with the Data Department lead to design and implement innovative solutions for efficiently processing large datasets using distributed computing.