Senior Data Engineer
Current- Stored and managed large volumes of data in AWS S3 ensuring data availability and durability for analytics and processing.
- Orchestrated over 30 data workflows with Apache Airflow, scheduling and managing complex ETL pipelines and data processing tasks, leading to a 35% increase in pipeline reliability.
- Leveraged AWS Redshift for high-performance data warehousing and analytics, optimizing query performance and reducing execution time by 50%.
- Implemented MLflow to manage the machine learning lifecycle, including experimentation, reproducibility and model deployment.