Software Engineer
CurrentLed the integration of 30+ complex data extraction pipelines and developed 10+ Airflow DAGs &scrapers, automating ETL processes to streamline data acquisition, boost efficiency, reduce costs,and enhance data accuracy.Led the design and implementation of full-stack components for internal and external dataanalysis tools, significantly improving data accessibility, and analytical capabilities, and driving highquality insights across the organizationAssisted in enhancing… Show more Led the integration of 30+ complex data extraction pipelines and developed 10+ Airflow DAGs &scrapers, automating ETL processes to streamline data acquisition, boost efficiency, reduce costs,and enhance data accuracy.Led the design and implementation of full-stack components for internal and external dataanalysis tools, significantly improving data accessibility, and analytical capabilities, and driving highquality insights across the organizationAssisted in enhancing ETL error handling by automating file storage in S3 during failed loadoperations and integrating SQS for retrying uploads to the database, improving data recovery.Contributed to migrating from a single job scheduler to a distributed system using Redis forjob management, optimizing thread management to improve ETL process efficiency.Collaborated with cross-functional teams to develop scalable solutions and train on newtechnologies and tools, earning 9 times “Performer of the Sprint” awards for driving innovationand excellence. Show less