o Created a Data-driven workflow for batch and near real-time to migrate the data from multiple sources (SQL-Servers/IoT Devices) via Azure Data Factory and Event hub to Azure Lake and process using Spark.o Created a Data Profiling framework that uncovers data quality issues in data sources, and what needs to be corrected in ETL using Sparko Created a Data Quality framework to filter out the unhealthy & clean data based on the quality checks o Using Azure Time-series insight for visualization of asset-based IoT data for ad-hoc analysis.o Created an Azure dashboard that provides a way to visualize data from multiple Azure resources.o Involved in all the Acquisition, Transformation, and Model phases of the project

o Worked on building a set of tools to create a data pipeline between the source data and the ingestion layero Design and Developed near real-time strategies to migrate data from RDBMS to S3/HDFSo Used real-time pipelines like Apache Kafka, and Amazon Kinesis for load balancing and to handle backpressure.o Implemented MR Job to parse data in multiple formats using HCatalog API and store in HDFS

o Migrated entire data from the On-premises cluster to the Google cloud platform and made the code platform agnostic. o Built utilities for a byte to byte table comparison, Data Lineage from Azkaban Logs, Data Copy utility between On-premises cluster and Google cloud platform thus saving the manual effort of 16 man-hours every week

o Designed specialized customer value offering for stores by analyzing demand based onneighborhood and spend informationo Used Hive and Pig scripts to create the insights table, used to generate reports for theanalysis

Learnt about the workflow of a hospital. Worked on HIS/RIS and other technological advancements in the healthcare field.