Mohan C Email & Phone Number

Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL.Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse.Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.Created Spark Structured Streaming Applications to consume data from EventHub and load into Delta tables after transformations.Built Spark Batch application to publish messages to Service Bus for SAP application integration with Mule Soft.Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the Sql Activity.Hands-on experience on developing SQL Scripts for automation purpose.Created Build and Release for multiple projects (modules) in production environment using Azure DevOps.Responsible for documenting the process and cleanup of unwanted data.Hands on experience in working on Spark SQL queries, Data frames, and import data from Data sources, perform transformations; perform read/write operations, save the results to output directory into HDFS.Worked in an Agile development environment in sprint cycles of two weeks by dividing and organizing tasks.Created Databricks Delta tables to store various data formats coming from different applications.Experience in working with Restful APIs.Responsible for modifying the code, debugging, and testing the code before deploying on the production cluster.

Attended requirement calls and worked with Business Analyst and Solution Architects to understand the requirements from clients. Analyzed the data flow from different sources to target to provide the corresponding design Architecture in Azure environment.Take initiative and ownership to provide business solutions on time.creating pipelines, data flows and complex data transformations and manipulations using ADF and PySpark with DatabricksIngested data in mini-batches and performs RDD transformations on data by using Spark Streaming API to performstreaming analytics in Data bricks.Created, provisioned different Databricks clusters needed for batch and continuous streaming data processing and installed therequired libraries for the clusters.Improved performance by optimizing computing time to process the streaming data and saved cost to company by optimizing thecluster run time.Perform ongoing monitoring, automation and refinement of data engineering solutionsLoaded different files from ADLS by using U-SQL scripts into target Azure Data warehouse.Worked on complex U-SQL for the data transformation and loading table and report generation.Designed and developed a new solution to process the NRT data by using Azure stream analytics, Azure Event Hub and Service BusQueue.Created Linked service to land the data from Caesars SFTP location to Azure Datalake.Created numerous pipelines in Azure using Azure Data Factory v2 to get the data from different source databases Informix, Sybase etcby using different Azure Activities like Move &Transform, Copy, filter, for each, Databricks etc.Created several Databricks Spark jobs with Pyspark to perform several tables to table operations.Working with complex SQL, Stored Procedures, Triggers, and packages in large databases from various servers.Experience in creating Data Lake Analytics account and Date Lake Analytics Job in Azure Portal using U-SQL Script.

Created complex Freeform’ s at Folder and Property level for data entry and maintenance.Undertake other software development project related tasks which may be reasonably expected.Analyzing the given documents, understanding and implementing it.Implemented the changes according to the change request document.Written many queries by using different sql objects, functions and keywords.Architected and Implemented complex ETL solutions for data migration and data staging using SSIS from various data sources (Excel,Flat, FTP, SQL Server).Responsible for writing T-SQL queries, designing SSIS packages, and SSRS reports, and analyzing existing stored procedures along withrecommending modifications for query efficiency.Participated in gathering User Requirements and System Specifications. Migrated DTS packages to SSIS packages and modified thepackages with new features of SQL SSIS 2012.Implemented performance tuning wherever required by creating required indexes, Eliminating spool space issues etc.Participate in peer-reviews, including, but not limited to design and development (coding).Participate and present status updates in each of the Stand-Up meetings on a regular basis.Solve complex problems in a time bound manner, document issues and fixes for easy system maintenanceAnalyzed the errors in error tables and reported to the client for restart ability.Did data reconciliation across source systems.Creating Unit test cases and implemented unit test results.Performance tuning, SQL query enhancements, code enhancements to achieve performance targets using Explain Plans.Involved in writing Fast load, Multi load and scripts to make loading, Updating.Answered Ad-hoc request of the client by using Functions, Analytical functions etc.

Export & import of objects (tables, schemas, views, packages) as per team’s requestsMonitoring Backups using RMAN utility and recovering the databases in case of any failures using the previous backups.Monitoring backup logs and moving archive logs to the tape as soon as certain space threshold limit reaches.Performing backups regularly using hot back up strategies for critical and cold backups for Non-critical databases.Worked on Oracle Enterprise Manager (OEM) to register the targets and monitor them from GUI and configuring threshold limits, alerts.Creating and maintaining different kinds of table space, data files, and space issue.Daily monitoring of the DB Sanity checks and storage checks either using SQL Developer or calling SQL scripts from Shell.Action on alerts and DBA errorsSpace management of the servers at OS level and DB level.Monitoring session and invalid objects and performing package deployments.Statistics gathering of DB and analysisSecurity Management by allocating proper privileges to users upon their requests.Involved in Upgrading Oracle database from oracle 9i to Oracle 10g.Altered the Table space sizes and resized Data files whenever any table space error occursUsed Oracle Export & Import utilities for refreshing schemas and tables of Dev and Test environments.Added users to SQL Servers and assigning roles to them.Experience in supporting SQL Server day to day tasks and backing up Databases for recovery.