• Worked with data transfer from on-premise SQL servers to cloud databases (Azure Synapse Analytics (DW) and Azure SQL DB). • Created Pipelines which were built in Azure Data Factory using Linked Services/Datasets/Pipeline/ to extract, transform, and load data from a variety of sources including Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and reverse. • Created CI-CD Pipelines using Azure DevOps • Used a blend of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics, gather, convert, and load the data from source systems to Azure Data Storage services. • Processed structured and semi-structured data into Spark Clusters using Spark SQL and DataFrames API. • Created Spark apps with Azure Data Factory and Spark-SQL for data extraction, transformation, and aggregation from various file formats in order to analyze and transform the data in order to reveal insights into consumer usage patterns. • Ingestion of data into one or more Azure Services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing of data in Azure Databricks. • Monitored the SQL scripts and modified them for improved performance using PySpark SQL. • Managed relational database service in which Azure SQL manages scalability, stability, and maintenance. • Integrated data storage options with Spark, notably with Azure Data Lake Storage and Blob storage. • Configured stream analytics, Event Hubs, and worked with Azure to manage IoT solutions. • Successfully executed a proof of concept for Azure implementation, with the wider objective of transferring on-premises servers and data to the cloud. • Proven track record of optimizing Spark application performance for optimal batch interval time, parallelism level, and memory optimization. • Used Hive queries to analyze huge data sets of structured, unstructured, and semi-structured data.

• Extensively used AWS Athena to import structured data from S3 into other systems such as Red Shift or to generate reports. • Worked with Spark to improve the speed and optimization of Hadoop's current algorithms. • Migrated an existing on-premises application to AWS. AWS services such as EC2 and S3 were used for data set processing and storage. Experienced in maintaining a Hadoop cluster on AWS EMR. • Created a Data Pipeline utilizing Processor Groups and numerous processors in Apache Nifi for Flat File, RDBMS as part of a Proof of Concept (POC) on Amazon EC2. • Migrated an existing on-premises application to AWS. AWS services such as EC2 and S3 were used for data set processing and storage. Experienced in maintaining a Hadoop cluster on AWS EMR. • The Spark-Streaming APIs were utilized to perform on-the-fly transformations and actions for the common learner data model, which gets data from Kinesis in near real-time. • Performed end-to-end architecture and implementation evaluations of different AWS services such as Amazon EMR, Redshift, S3, Athena, Glue, and Kinesis. Hive We created external table schemas for the data being processed as the primary query engine of EMR. • Created Apache presto and Apache drill configurations on an AWS EMR (Elastic Map Reduce) cluster to integrate different databases such as MySQL and Hive. This allows for the comparison of outcomes such as joins and inserts on many data sources controlled by a single platform. • AWS RDS (Relational database services) was created to act as a Hive meta store, and metadata from 20 EMR clusters could be integrated into a single RDS, preventing data loss even if the EMR was terminated. • Developed and implemented ETL pipelines on S3 parquet files in a data lake using AWS Glue. • Developed a cloud formation template in JSON format to utilize content delivery with cross-region replication using Amazon Virtual Private Cloud.

• Developed distributed high-performance systems with Spark and Scala. • Created Scala apps for loading/streaming data into NoSQL databases (MongoDB) and HDFS is preferred. • Performed T-SQL tuning and query optimization for and SSIS packages. • Developed distributed algorithms for detecting and successfully processing data trends. • Created an SSIS package to import data from SQL tables into various Excel sheets. • Used Spark SQL to pre-process, clean, and combine big data sets. • Developed the SQL server database system to optimize performance. • Performed migration of databases from conventional data warehouses to spark clusters. • Performed frequent cleaning and integrity tests, you can ensure that the data warehouse was only loaded with high-quality entries. • Developed SQL queries to extract data from existing sources and validate format correctness. • Created automated tools and dashboards for collecting and displaying dynamic data.

• Power BI Based Patient Registration Monitoring Application • Created a Power BI data model based on analysis of the end-user workflow data provided by the client. • Imported data from SQL Server DB, Azure SQL DB to Power BI to generate reports. • Developed analysis reports and visualization using DAX functions like table function, aggregation function, and iteration functions. • Produced custom visualizations for the client to monitor all patient registration-related workflow, thus giving excess to their supervisor team to monitor high error rate departments • Presented application to Hospital’s Executive Team and put the application into a production system for them to use SQL & Tableau Projects • Designed, developed, tested, and maintained Tableau functional reports based on client requirements • Extracted data from SQL Server database tables into flat data files and Excel sheets using table export and BCP for easy data migration • Developed advanced SQL queries with multi-table joins, group functions, subqueries, set operations, and T-SQL stored procedures, user-defined functions (UDFs) for data analysis

• Involved in designing physical and logical data model using ERwin Data modeling tool.• Designed the relational data model for operational data store and staging areas, Designed Dimension & Fact tables for data marts.• Extensively used ERwin data modeler to design Logical/Physical Data Models, relational database design.• Created Stored Procedures, Database Triggers, Functions and Packages to manipulate the database and to apply the business logic according to the user's specifications.• Created Triggers, Views, Synonyms and Roles to maintain integrity plan and database security.• Creation of database links to connect to the other server and Access the required info.• Integrity constraints, database triggers and indexes were planned and created to maintain data integrity and to facilitate better performance.• Used Advanced Querying for exchanging messages and communicating between different modules.• System analysis and design for enhancements Testing Forms, Reports and User Interaction.