Shiva Kumar Email & Phone Number

Texas, United States

• Involved in design of Data-Warehouse using data modeling techniques like Star-Schema and converted data from various sources to SQL Tables.• Create SQL Scripts, Notebooks for the data transformation in Azure Synapse Analytics.• Working with Serverless and Dedicated SQL servers, Creating Apache Spark pools and scaling of the nodes based on data.• Created health check pipelines for data maintenance.• Serve as Scrum Master and Lead daily stand-ups to work closely with Product Owners to develop, maintain project backlogs, and to ensure that the highest business value user stories are delivered to the customers.• Production Failure Handling.• Use Web activity to get response from REST API’s and use that response in making business decisions.• Involved in ETL development which includes creating the Integration Runtime’s, Linked Services, datasets and pipelines using Azure Cloud services to establish a connection with On-prem data sources.• Pull the data to Azure Data Lake Storage from On-Premises SQL Server, perform transformations on the Raw-Data using SCALA, Python Scripting & Spark SQL in Databricks and build deltalakehouse table to store the transformed data.• Creation of Databricks Notebooks, Load the data into dataframes and transform it using python Scripting.• Create Secret scopes for accessing the data, Cluster optimizations, Understand Databricks architecture to handle the driver and worker nodes, Scheduling the jobs to run them with the help of Job clusters.• Build the pipelines in ADF and Azure Synapse Analytics which automates the process of extraction of raw-data from on-premises source using various activities like copy, lookup, foreach, if, execute pipeline, notebook, stored proc activities.• Creating ADF triggers to schedule the pipelines to run at frequencies.• Deploy the code to different environments through dev-ops using CI/CD Pipelines.• Build Reports using Power BI.• Develop Logic-App’s to alert the end users about data load status.

Bengaluru, Karnataka, India

• Analyze the existing system and provide feasible solution as per the requirements.• Serve as Scrum Master and Lead daily stand-ups to work closely with Product Owners to develop and maintain project backlogs, and to ensure that the highest business value user stories are delivered to the customers.• Involved in complete software development life cycle (SDLC) process by analyzing business requirements and understanding the functional workflow of information from source systems to destination systems.• Involved in ETL development which includes creating the Integration Runtime’s, Linked Services, datasets to establish a connection with On-prem data sources.• Using the Azure cloud services like Azure Data Factory, Azure Data Lake Storage, and Azure Databricks pull the data from On-Prem SQL Server, perform transformations on the Raw-Data using SCALA, Python & Spark SQL in Databricks.• Build the pipelines in ADF, which automates the process of extraction of raw-data from on-prem source using various activities like copy, lookup, foreach, if, execute pipeline, notebook, stored proc activities.• Creating ADF triggers to schedule the pipelines to run at frequencies.• Deployment of Databricks using Kubernetes.• Deploy the code to different environments through dev-ops using CI/CD Pipelines, Devops.• Build Reports using Power BI.• Develop Logic-App’s to alert the end users about data load status.

Bengaluru, Karnataka, India

• Involved in design of Data-Warehouse using Snow-flake Schema data model and converted data from various sources to SQL Tables.• Strong experience in Oil & Gas Data Warehouse modules.• Develop ETL pipelines for data migration.• Transformation of data using Python & PySpark.• Created health check pipelines for data maintenance.• Enhance the existing Pipelines for optimization.• Write complex SQL queries as per business requirements.• Extraction and Transformation of data from a wide variety of sources such as XML data files, flat files, and relational data sources and then load the data into one or more destinations using SSIS.• Creation of reports using SSRS after the data is loaded to the target.• Performance tuning of SQL queries to improve the run times, which enhances the performance of the queries.• Production Failure Handling.• Use Web activity to get response from REST API’s and use that response in making business decisions.• Creation of PowerBI reports and publishing it to PowerBI Service and setup the dataset refresh. • Data Validation and Monitoring to remove the duplicates.

Bengaluru, Karnataka, India

• Design a Data Warehouse for the system.• Since ebay is a retail domain, used to handle big data which is around millions of records.• Monitoring and transformation of big data using ETL Pipelines and Python Scripting.• Incorporate new enhancements and fix bugs.• Perform data cleansing, enrichment, mapping tasks and automated data validation process to ensure meaning full and accurate data was reported efficiently.• Create user defined functions to meet specific purpose.• Testing of ETL Pipelines includes data validation.• Monitoring of production pipelines and raise defects in case of failures.• Performance tuning of existing Stored Procedures.• Designing of OLAP & OLTP Database Solutions.• Tuning of Report queries to improve the report load time.• Performed testing of all reports for the final deployment.

Bengaluru, Karnataka, India

• Performance tuning of existing Microsoft SQL Queries to improve query time.• Building and working with Relational Databases.• Data monitoring to avoid data slippage and duplicates.• Design, Coding/Development, Incorporate New Enhancements to the existing project and Fix defects.• Performance tuning of database objects like Stored Procedures , Functions..• Develop Unit and integrated Test-Cases for data validation.• Perform testing of data and raise defects in case of data mismatch.• POC preparation for implementation & adaptation of new ideas into work.