Senior Data Engineer
CurrentI focused on optimizing data infrastructure, using tools like Apache Spark, Scala, and Databricks to cut processing time by 50% for large datasets. I designed scalable data warehousing solutions on AWS, fine-tuning ETL processes with Airflow and Python for faster and more accurate data retrieval. I also led real-time data integration projects using Kafka and Spark, which significantly boosted customer insights and decision-making. Automation was a big part of my work, where I streamlined… Show more I focused on optimizing data infrastructure, using tools like Apache Spark, Scala, and Databricks to cut processing time by 50% for large datasets. I designed scalable data warehousing solutions on AWS, fine-tuning ETL processes with Airflow and Python for faster and more accurate data retrieval. I also led real-time data integration projects using Kafka and Spark, which significantly boosted customer insights and decision-making. Automation was a big part of my work, where I streamlined workflows with Apache Airflow to ensure reliable data delivery. Throughout, I collaborated with cross-functional teams and mentored junior engineers in data engineering best practices. Show less