Data Engineer
Current• Automated 80% of the tasks using SQL and Python scripts, saving the team roughly 10 hours of manual work each week.• Created and managed robust data pipelines using AWS Glue ETL, optimizing data flow and processing efficiency by 45% while ensuring seamless integration and reliability.• Applied advanced techniques to optimize data processing Using PySpark and Spark-Sql, resulting in a 45% reduction in processingTime and a 30% increase in data throughput efficiency.• Leveraged Spark to automate the data cleaning process, achieving high-performance data handling and significantly reducing processing times.• Utilized AWS services including EC2, S3, Lambda, Glue, and Athena to build scalable data processing solutions, reducing data Processing times by 50% and enhancing data accessibility and integration efficiency by 40%.• Leveraged Snowflake for seamless data storage and pipeline integration, improving data access efficiency by 40% and reducing data handling time by 30%.• Applied advanced data modeling techniques, including dimensional, physical and logical data modeling, to optimize database Structures for performance and scalability.• Orchestrated data pipelines using Apache Airflow on AWS, ensuring reliable and scalable data processing workflows that support critical business operations.• Designed and deployed interactive Tableau dashboards, enhancing data visualization and insight discovery by 40%, leading to Improved decision-making processes.