Involved in gathering requirements, system analysis, design, development, testing and deployment.Design & implement migration strategies with Azure suite: Azure SQL Database, Azure Data Factory (ADF) V2, Azure key Vault, Azure Blob Storage.Created end to end Data pipelines using ADF services to load data from On-prem to Azure SQL server for Data orchestration.Build scalable and reliable ETL pipelines to pull large and complex data from different systems efficiently.Worked on building the data pipeline using Azure Service like Data Factory to load the data from Legacy, SQL server to Azure Data warehouse using Data Factories and Databricks Notebooks.Created Pipeline’s to extract data from on premises source systems to azure cloud data lake storage.Extensively worked on copy activities and implemented the copy behavior’s such as flatten hierarchy,preserve hierarchy and Merge hierarchy.Implemented Error Handling concept through copy activity.Exposure on Azure Data Factory activities such as Lookups, Stored procedures, if condition, for each, SetVariable, Append Variable, Get Metadata, Filter and wait.Configured the logic apps to handle email notification to the end users and key shareholders with the help of web services activity.Create and maintain optimal data pipeline architecture in cloud Microsoft Azure using Data Factory andAzure Databricks.Exposed transformed data in Azure Spark Databricks platform to parquet formats for efficient data storage.Analyzed data where it lives by Mounting Azure Data Lake and Blob to Databricks.Extensively worked on Azure Data Lake Analytics with the help of Azure Data bricks to implement SCD-1,SCD-2 approaches.Created and configured a new event hub with the provided event hubs namespace.Writing PySpark and spark SQL transformation in Azure Databricks for business rule implementationInvolved in tuning spark Jobs and worked on spark Query Optimization.

Involved in Azure cloud, App services, Azure SQL Database, Azure Blob storage, Azure Functions, Virtual machines, Azure AD, Azure Data Factory, event hub and event Queue.

Creating pipelines, data flows and complex data transformations and manipulations using ADF and PySpark with Databricks.

⮚ Involved heavily in setting up the CI/CD pipeline using Jenkins, Maven, Nexus, GitHub, and AWS.⮚ Developed data pipeline using flume, Sqoop, pig and map reduce to ingest customer behavioral data and purchase histories into HDFS for analysis.⮚ Used Spark-SQL to load JSON data and create schema RDD and loaded it into Hive tables handled structured data using Spark SQL⮚ Used HIVE to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.⮚ The Hive tables created as per requirement were internal or external tables defined with appropriate static and dynamic partitions, intended for efficiency.⮚ Implemented the workflows using Apache Oozie framework to automate tasks.⮚ Developing design documents considering all possible approaches and identifying best of them.⮚ Written Map Reduce code that will take input as log files and parse the and structures them in tabular format to facilitate effective querying on the log data.⮚ Developed scripts and automated data management from end to end and sync up b/w all the Clusters.⮚ Implemented Fair schedulers on the Job Tracker to share the resources of the cluster for the Map Reduce jobs given by the users. Environment: Cloudera CDH 3/4 Distribution, Tibco, HDFS, MapReduce, Hive, Oozie, Pig, Shell Scripting, MySQL.

⮚ Involved as a developer for the commercial business group data warehouse.⮚ Development of source data profiling and analysis, reviewed the data content and metadata would facilitate data mapping and validate assumptions that were made in the business requirements.⮚ Extracted data from relational databases MySQL and flat files.⮚ Designed complex mappings using Source Qualifier, Joiners, Lookups (Connected and Unconnected) and Expression, Filters, Router, Aggregator, Sorter, Update Strategy, Stored procedure and Normalizer transformations.⮚ Efficient in Dimensional Data Modeling for Data Mart design, identifying Facts and Dimensions, and developing, fact tables, dimension tables, using Slowly Changing Dimensions (SCD).⮚ Ensured the data consistency by cross-checking sampled data upon migration between the database environments.⮚ Developed a process to extract the source data and load it into the flat files after cleansing, transforms and integrating. ⮚ Designed SSIS packages to Extract, Transfer and load the (ETL) existing data into SQL server from different environments for the SSAS cubes.⮚ Shared data outside using Snowflake to quickly set up to share data without transferring or developing pipelines.⮚ Provided SSRS and SSIS support for internal IT projects requiring report developments.⮚ Involved in System Integration Testing (SIT) and User Acceptance Testing (UAT).Environment: Informatica 8.6.1, SQL Server 2005, RDBMS, Fast load, FTP, SFTP.

⮚ Create and maintain database for Server Inventory, Performance Inventory.⮚ Worked in Agile Scrum Methodology with daily stand up meetings, great knowledge working with Visual SourceSafe for Visual studio 2010 and tracking the projects using Trello.⮚ Generated Drill through and Drill down reports with Drop down menu option, sorting the data, and defining subtotals in Power BI.⮚ Used Data warehouse for developing Data Mart which for feeding downstream reports, development of User Access Tool using which users can create ad-hoc reports and run queries to analyze data in the proposed Cube.⮚ Created logical and physical designs of the database and ER Diagrams for Relational and Dimensional databases using Erwin.⮚ Deployed the SSIS Packages and created jobs for efficient running of the packages. ⮚ Expertise in creating ETL packages using SSIS to extract data from heterogeneous database and then transform and load into the data mart. ⮚ Involved in creating SSIS jobs to automate the reports generation, cube refresh packages. ⮚ Thorough knowledge of Features, Structure, Attributes, Hierarchies, Star and Snow Flake Schemas of Data Marts.⮚ Experienced with SQL Server Reporting Services (SSRS) to author, manage, and deliver both paper-based and interactive Web-based reports. ⮚ Developed stored procedures and triggers to facilitate consistent data entry into the database.Environment: Windows server, MS SQL Server 2014, SSIS, SSAS, SSRS, SQL Profiler, Dimensions, snowflake, Power BI, Performance Point Server, MS Office, SharePoint.