Senior Big Data Engineer
CurrentLed big data analytics, predictive analytics, and machine learning initiatives, primarily employing AWS services and Snowflake for data warehousing and business intelligence solutions. created and enhanced ETL processes for data aggregation, transformation, and analysis utilizing Python, Scala, and Apache Spark. Automated data pipelines using Apache Airflow and Shell scripting to streamline production processes. competent in deploying data quality measures, profiling a variety of data sources, and integrating and migrating data utilizing AWS S3, DynamoDB, and Snowflake. Capable of publishing code to AWS EMR using CI/CD pipelines using Jenkins and maintaining version control with Code cloud.