Sourabh Magadum Email and Phone Number

● Utilized Java, Spring Boot, Apache Spark, and Apache Kafka to create and manage datapipelines on Microsoft Azure, resulting in efficient data processing● Developed multiple microservices to manage data flow from sources to consumers, includinga job trigger service to monitor and log new data paths in a PostgreSQL table, and a jobmanager service to orchestrate Spark jobs for efficient data processing● Developed an ETL Reconciliation application to vigilantly monitor data completeness byintegrating the data pipeline with an API, enabling real-time status updates on data transfers● Improved ETL pipeline process time by 70% through strategic optimization of Spark jobs andheightened parallelism during peak loads, leveraging Azure cluster scheduled scaling. Thisinitiative resulted in a notable €8,000 reduction in monthly infrastructure expenditures● Actively contributed to On-Premise to Azure migration, Dockerizing, and migrating services toAzure Kubernetes Services● Developed REST APIs that fetch data from HDFS and Postgres tables for visualizations andprocess monitoring● Improved security and visibility by creating a Java library to retrieve data from HDFS for theclients, and put in place a system to track users of the data by storing their metadata in adatabase● Automated Azure HD Insight cluster recreation by creating a DevOps pipeline

• developed a Fare Mapping model for the airline, optimizing pricing through Fare Basis Codes and Fare brand analysis • Developed Python scripts for Fare Mapping, Fare Basis Code decoding, and web scraping of Fare Brands. • Created a customer-focused ChatBot using Amazon Lex and Lambda Function for enhanced client interaction.