Staff Data Engineer
Current• Architected and implemented Airflow DAG management framework enabling complex workflow logic and actionable alerting.• Designed data pipeline that automatically handles file structure changes; eliminating the need for manual updates and configurations. • Lead developer in developing ETL pipeline using Python, Airflow, Glue and AWS Lambdas; enabling quick onboarding of 1,000+ customers.• Subject-matter-expert with several critical services, training teammates as appropriate to reduce knowledge silos.• Optimized SQL queries, indices and data structures for performance and scalability.• Maintained infrastructure as code (IaC) with Terraform, Docker and Jenkins.