Senior Technical Lead
Current- Promoted to Technical Lead, overseeing a team of big data engineers while continuing to contribute as a hands-on developer, ensuring project success through technical guidance, code reviews, and architectural decisions.
- Developed and implemented automation tools using Scala, Presto JDBC, Hive, and Spark to enhance Flare pipeline performance
- Developed comprehensive ETL Automation Application for scheduled daily, weekly, and monthly ETLs, incorporating multi-query summaries, data trend checks, status notifications, and automated file transfer to edge nodes.
- Led migration of CVM campaign generation process from Oracle to HDFS, reducing processing time by 80%. Converted key summaries from Oracle to Presto, managed the migration team, and integrated the process with the ETL.
- Designed and built API Data Extractor to process JSON data from various endpoints.
- Engineered Partition Rewrite Application to optimize queries performances and resolve Hadoop small file issues