Responsible for maintaining quality reference data in source by performing operations such as cleaning,transformation and ensuring Integrity in a relational environment by working closely with the stakeholders& solution architect. Designed and developed Security Framework to provide fine grained access to objects AWS S3 usingAWS Lambda, DynamoDB. Performed end- to-end Architecture & implementation assessment of various AWS services like AmazonEMR Redshift, S3. Used AWS EMR to transform and move large amounts of data into and out of other AWS data stores anddatabases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB. Install and configure Apache Airflow for AWS S3 bucket and created DAGs to run the Airflow. Prepared scripts to automate the ingestion process using Pyspark and Scala as needed through varioussources such as API, AWS S3, Teradata and Redshift. Created multiple scripts to automate ETL/ ELT process using Pyspark from multiple sources. Developed Pyspark scripts utilizing SQL and RDD in spark for data analysis and storing back into S3. Developed Pyspark code to load from stg to hub implementing the business logic. Developed code in Spark SQL for implementing Business logic with python as programming language. Worked on Sequence files, Map side joins, bucketing, partitioning for hive performance enhancement andstorage improvement. Wrote, compiled, and executed programs as necessary using Apache Spark in Scala to perform ETL jobsingested data.

Developed a deep understanding of the data sources, implemented data standards, maintained data quality,and mastered data management. Build Complex distributed systems involving huge amounts of data handling, collecting metrics buildingdata pipeline, and Analytics. Extract Transform and Load data from Sources Systems to Azure Data Storage services using acombination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Involved in Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL,Azure DW) and processing the data in Azure Databricks. Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization ofdata. Understand the current Production state of the application and determine the impact of newimplementation on existing business processes. Implemented data governance practices on Azure, including data security, privacy, and compliance withregulations such as GDPR and HIPAA. Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load datafrom different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool, andbackward. Involved in developing Spark applications using PySpark and Spark-SQL for data extraction,transformation, and aggregation from multiple file formats for analyzing & transforming the data touncover insights into customer usage patterns. Created several Databricks Spark jobs with PySpark to perform several tables-to-table operations. Create and maintain optimal data pipeline architecture in cloud Microsoft Azure using Data Factory and Azure Databricks

Actively Participated in all phases of the Software Development Life Cycle (SDLC) from implementationto deployment. Responsible for building scalable distributed data solutions using Hadoop. Responsible for Cluster maintenance, adding and removing cluster nodes, Cluster Monitoring andTroubleshooting, Managing, and reviewing data backups & log files. Responsible for managing the test data coming from different sources. Analyzed data using Hadoop components Hive and Pig. Load and transform large sets of structured, semi structured, and unstructured data using Hadoop/Big Data concepts. Involved in importing and exporting the data from RDBMS to HDFS and vice versa using Sqoop. Worked on developing ETL processes (Data Stage Open Studio) to load data from multiple data sourcesto HDFS using FLUME and SQOOP, and performed structural modifications using Map Reduce, HIVE. Developing Spark scripts, UDFS using both Spark DSL and Spark SQL query for data aggregation,querying, and writing data back into RDBMS through Sqoop. Strong understanding of Partitioning, bucketing concepts in Hive and designed both Managed andExternal tables in Hive to optimize performance. Interacted with business partners, Business Analysts, and product owner to understand requirements andbuild scalable distributed data solutions using the Hadoop ecosystem. Developed Spark Streaming programs to process near real time data from Kafka, and process data withboth stateless and stateful transformations. Experience in report writing using SQL Server Reporting Services (SSRS) and creating various types ofreports like drill down, Parameterized, Cascading, Conditional, Table, Matrix, Chart and Sub Reports. Wrote Oozie scripts and set up workflow using Apache Oozie workflow engine for managing andscheduling Hadoop jobs. Used Hive to analyze data ingested into HBase by using Hive-HBase integration and compute variousmetrics for reporting on the dashboard.

Gather Requirements that are to be incorporated into the system. Extensively worked on the analysis of Tables in both Legacy Data store and new data store. Extensively worked on the analysis of Columns in mapping tables for both Legacy Data store and newdata store. Initialized utilization of Data Warehouse ETL software during conversion of data to Oracle DB. Developed the complete documentation of the project based on the analysis of tables and Columns. Created DDL scripts to create, alter, drop tables, views, synonyms and sequences. Worked on SQL Tables, Records and Collections. Wrote SQL Procedures, Functions, and Triggers for Insert, Update and Delete transactions and optimizedfor maximum performance. Extensively worked on the Database Triggers, Stored Procedures, Functions and Database Constraints. Developed SQL queries to fetch complex data from different tables in remote databases using databaselinks. Used ETL process to identify the new or the changed data in order to make better decisions in the project. Participated in Performance Tuning of SQL queries using Explain Plan to improve the performance of theapplication. Source data residing in Excel formats are exported to flat files and then accessed via Oracle external tablesin order to load into the staging schema, at which point all source data can be efficiently transformed andmigrated to the target schema. Extracted data from Flat files using SQL*LOADER. Developed Unix Shell Scripts for loading data into the database using SQLCreated partitions on the tables to improve the performance. Participated in application planning, design activities by interacting and collecting requirements from the end users.

Generated database monitoring and data validation reports in SQL Server Reporting Service (SSRS). Created partition table for very large database for good performance. Designed dynamic SSIS Packages to transfer data crossing different platforms, validate data duringtransferring, and archived data files for different DBMS. Responsible for writing SQL queries, stored procedures, views, triggers, T-SQL and DTS/SSIS Deployed SSIS packages and Reports to Production Servers. Worked on Varian EMR Integration to Track Data overtime and improve overall quality of care in apractice. Involved in creation of Data Warehouse Cubes in SQL Server Analysis Service (SSAS). Migrated data using ETL Tool in SSI. Maintained system monitoring jobs and provide high availability support. Worked on Migration of packages from DTS using SQL Server Integration Service (SSIS). Participated Databases Architecture and data modeling design. Reported all events and requirements through established reporting mechanisms in SSRS. Generated test data and tested database to meet the functionalities deliverables in the projectdocumentation and specifications. Designed and developed OLAP using MS SQL Server Analysis Services (SSAS). Designed and developed MS SQL Server Reporting Services (SSRS) under SQL Server 2008. Generated periodic reports based on the statistical analysis of the data using SQL Server ReportingServices (SSRS). Created Drill-down, Parameterized and Sub-Reports using RDL and promoted RDLs to Reporting8Service Server (SSRS).