- Involved in creating ETL pipelines to ingest data into Azure data lake.- Experience in creating Airflow DAGS for automating ETL pipelines.- Experience in creating helm chart and terraform sprits to deploy Applications.- Worked on creating POC’s on Running spark on Kubernetes for streaming jobs.- Experience in creating Prometheus rules for metric monitoring.- Worked on creating Helm charts Template which will be used across the organization.- Worked on creating Multiple spark UDF’s for optimizing spark execution and to reduce IO.- Experience in productionalize and optimizing data science models.- Used Kafka connect for reading CDC feed from MySQL, mongo DB, Sql-server, Cosmos DB in real timeinto Kafka.- Used spark on K8’s to read data from Kafka and write into Azure data-lake.- Pioneered a scalable end-to-end Machine Learning pipeline for Product Markdowns, includingscheduled data processing, model training, and predictions accessible through an on-premisescustom UI. Certain brands were selected using the model, and exceeded sales lift expectations(Python, Airflow, Spark, Databricks, Kubernetes, Helm, Flask, PostgreSQL).- Updates and monitors batch ETL processes for Foot Locker's Data Analytics Platform, whichtransfer various data sources into our centralized Azure Data Lake. This includes various bugfixes, backfills, and schema changes for our cloud database (Azure, Airflow, Spark, Kafka).

-Hands on experience in installing, configuring, and using Hadoop ecosystem components likeHadoop MapReduce, HDFS, HBase, Hive, Spark, Sqoop, Pig, Zookeeper and Flume.- Data warehouse, Business Intelligence architecture design and develop. Designed the ETLprocess from various sources into Hadoop/HDFS for analysis and further processing of datamodules.- Designing and Creating Azure Data Factory (ADF) extensively for ingesting data from differentsource systems like relational and non-relational to meet business functional requirements.- Extract Transform and Load data from Sources Systems to Azure Data Storage services using acombination of Azure Data Factory, Spark SQL Azure Data Lake Analytics.- Data Ingestion to at least one Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, AzureDW) and processing the data in In Azure Databricks.- Created, provisioned numerous Databricks clusters needed for batch and continuous streamingdata processing and installed the required libraries for the clusters.-Extensively chipped on Spark Context, Spark-SQL, RDD's Transformation, Actions and DataFrames.-Developed custom ETL solutions, batch processing and real-time data ingestion pipeline to movedata in and out of Hadoop using Pyspark and shell scripting.

- Responsible for developing and modifying the existing service layer based on the businessrequirements.- involved in designing & developing web-services using SOAP and WSDL.- Created tables, stored procedures in SQL for data manipulation and retrieval, DatabaseModification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle 9i.- Involved in integration testing the Business Logic layer and Data Access layer.- Designed user-interface and checking validations using JavaScript.- Involved in design of JSP's and Servlets for navigation among the modules.- Designed cascading style sheets and XML also performed client-side validations with java scriptusing Business Objects, XML, and JDBC.- Developed various EJBs for handling business logic and data manipulations from database.