Data Engineer
Current• Collaborated with engineering teams to design and develop Python-based ETL processes for data mapping and transformation.• Managed the full lifecycle of the Learning Management System (LMS), including system configuration, user setup, and ongoing system administration to ensure smooth functionality and user experience.• Integrated SQL, R, Python, and Excel functionalities into the Learning Management System (LMS), enabling advanced data analysis and reporting capabilities for course performance, learner progress, and engagement metrics.• Migrated data between HDFS and AWS S3 and optimized AWS Glue and Lambda jobs for data integration.• Developed Spark and Python-based data pipelines to load and transform data for analytics using Redshift and PostgreSQL.• Performed tuning on Spark jobs, scheduled jobs using CTRL M, and deployed clusters on AWS EMR.