Data Engineer
Current- Achievements:- Reduced costs by 75% by developing a new ETL and data ingestion pipeline on AWS, including data quality.- Delivered reports 32x faster by implementing an automated tool that generates custom reports..
- Develop and maintain a comprehensive ETL and data ingestion pipelines on AWS, by using Airflow and Spark, for Bemobi's fraud prevention team, efficiently processing over half a million transactions daily with opsgenie.
- Developed and implemented an automated reporting solution that schedules, generates, processes, and delivers customized reports, which are also available on a centralized page with user access control.
- Developed and maintained CI/CD pipelines using Terraform (IaC)
- Maintains a custom docker image used in our analyses
- Developed a Private Python package, completely integrated into our custom docker image and pipelines.