Senior Data Engineer
Current• Stored and managed large volumes of data in AWS S3 ensuring data availability and durability for analytics and processing.• Orchestrated over 30 data workflows with Apache Airflow, scheduling and managing complex ETL pipelines and data processing tasks, leading to a 35% increase in pipeline reliability.• Leveraged AWS Redshift for high-performance data warehousing and analytics, optimizing query performance and reducing execution time by 50%.• Implemented MLflow to manage the machine learning lifecycle, including experimentation, reproducibility and model deployment.