Senior Etl Developer
Current• Designed multiple systems that replaced manual processes. These systems took disparate files from clients and transformed the data for consumption by multiple integrated systems. The systems reduced the business workforce by 10 resources and reduced customer data integration time by 99%. Every file is processed within an hour of receipt. Build in data quality and traceability by including logging and auditing as part of the system. (Penthao and MoveIt)• Managed a team of 5 offshore data engineers and collaborated with project and program management on project priorities, timelines, estimations and requirements. • Worked as a change agent directly with CTO and Senior Leadership Team on system design and data analysis to solve for new data regulations and contract requirements (CPRA, CCRA, Home Depot and State of Georgia).• Created AWS pipelines using Airflow, Kafka, Elastic Search, Pyspark, Greenplum, and S3 to maintain data warehouse.• Established and Maintained Pentaho as the company standard ETL platform supporting 10,000 running jobs daily. With 99.9% availability. Responsible for all planning and testing of platform upgrades and patches.• Established and Maintained MoveIt as the company standard File Exchange platform supporting internal and external automated file movement (90,000 files/day). Responsible for all planning and testing of platform upgrades and patches.• Created coding standards, system integration patterns, reusable objects to increase developer velocity and improve customer integration experience.