System Engineer
Current- Developed and maintained MWAA jobs, ensuring seamless data workflows across multiple platforms.- Enhanced Python/PySpark scripts to optimize operations, resulting in continuous data flow and reduced bottlenecks.- Implemented CI/CD pipelines using Git and Bitbucket, streamlining code deployment and minimizing manual intervention.- Performed detailed data analysis and troubleshooting to resolve production issues, ensuring a stable and efficient environment.- Upgraded critical… Show more - Developed and maintained MWAA jobs, ensuring seamless data workflows across multiple platforms.- Enhanced Python/PySpark scripts to optimize operations, resulting in continuous data flow and reduced bottlenecks.- Implemented CI/CD pipelines using Git and Bitbucket, streamlining code deployment and minimizing manual intervention.- Performed detailed data analysis and troubleshooting to resolve production issues, ensuring a stable and efficient environment.- Upgraded critical applications in PySpark, using pandas DataFrame in EMR clusters to minimize downtime and increase operational reliability.- Collaborated on knowledge transfer (KT) processes, maintaining detailed documentation for future reference and team efficiency. Show less