Data Engineer
Current• Developed and managed various data pipelines using AWS Glue to automate data ingestion, transformation, and loading processes, improving data processing efficiency by 30%.• Engineered robust ETL/ELT processes utilizing AWS Data Pipeline, enhancing data flow and performance, which led to a 30% increase in data processing speed and quality.• Leveraged Amazon Redshift as a data warehouse to consolidate and analyse data from various sources, improving query performance by 20% and reducing storage costs.• Migrated the data aggregation layer from AWS Glue to using Data Build Tool (DBT), resulting in up to 90% cost savings and improved query performance.