Data Engineer
CurrentAt Banner Health, I spearheaded the development of scalable data pipelines using Apache Kafka, Spark, and AWS for real-time patient data processing. I designed a distributed data processing framework with Hadoop and Hive, automated ETL processes using Airflow, and established a centralized data lake on AWS S3 for advanced analytics. I implemented real-time analytics with Spark Streaming and Kafka, migrated legacy systems to AWS Redshift, and ensured HIPAA compliance through robust data governance. I optimized critical data pipelines for performance, collaborated with cross-functional teams to deliver data-driven solutions, and mentored junior engineers in best practices. Additionally, I developed and maintained robust data governance and data quality processes, ensuring compliance with HIPAA regulations and industry best practices for handling sensitive healthcare data. I also conducted performance tuning and optimizations for critical data pipelines, resulting in significant improvements in data processing times and resource utilization.