Data Engineer
Current- Established and maintained an ETL pipeline using Informatica PowerCenter to extract, transform, and load data from multiple sources into a data warehouse, ensuring data accuracy and consistency.
- Accomplished a complex data processing workflow consisting of multiple stages (data extraction, transformation, loading) using AWS Pipeline, improving data pipeline reliability and 40% reduction in data processing.
- Optimized Lambda functions for cost efficiency, achieving a 20% reduction in execution costs through code refactoring and best practices.
- Migrated data aggregation layer from legacy services to Snowflake unitizing Data Build Tool (DBT) models resulting in up to 70% cost savings and improved query performance.
- Orchestrated complex data pipelines with 3 stages using AWS Step Functions to automate data movement and transformation tasks, ensuring reliable data flow.
- Implemented a data warehouse on Databricks using Delta Lake, improving data query performance by 40% compared to the previous Teradata-based solution.