Vamsi Krishna B Email & Phone Number

Glen Allen, Va, Us

Dallas, Texas, United States

Implementation of MSAL for authentication purpose so that authentication to Azure services from Databricks possiblewithout interruptions like expiration of Secrets and keys.Transformed data using azure Databricks with help of Pyspark.• Migrated the Storage accounts from V1 to V2, without any interruption in on going dataflow.• Utilized Delta Live Tables to efficiently load data into corresponding Delta tables, ensuring automated management ofdata lineage and dependencies.• Developed Databricks Delta tables for efficient data ingestion from multiple sources, implementing Medallion architectureprinciples to enhance data cleaning and transformation processes.• Implemented row level security for few delta tables using Unity catalog in Azure Databricks

Bengaluru, Karnataka, India

• Involved in loading and transforming large sets of Structured and Semi-Structured data and analyzed them by running Hive queries.• Spearheaded a big data project including requirement gathering, design, data collection, data modeling, data reporting, and unit testing and able to deliver it within the stipulated timeline.• Converted existing MapReduce programs to Spark to cleanse the data in HDFS obtained from multiple data sources to make it suitable for ingestion into Hive for analysis.• Optimized Hive analytics SQL queries, created tables/views, written custom UDFs and Hive based exception processing.Worked on Dimensional and Relational Data Modeling using Star and Snowflake Schemas, OLTP/OLAP system, Conceptual, Logical and Physical data modeling using Erwin.• Involved in transforming the large data from Teradata to HDFS and vice versa using Sqoop incremental imports.• Exported the analyzed data to the relational databases using Sqoop and to generate reports for the BI team.• Collaboratively worked with business analysts to translate business insights into decisions and actions.

Bengaluru, Karnataka, India

Worked on re-platforming of medical claims data from legacy systems to bigdata environment.∗ Imported raw data from various data sources using SQL queries for data mining tasks such as joins and developedoptimised sql queries.∗ Intensively used spark and SQL to analyze the claims data submitted by the patients and build data pipelines to processit into final warehouse hosted on hive with hadoop as the underlying storage∗ Ingesting the data from RDBMS to data lake. Apply the transformations and load the data to hive tables.∗ Implemented complex transformation logic in Apache Spark using RDDs and DataFrames, and optimized Sparkconfiguration parameters to enhance performance.∗ Ensure that the workflow has to be run on daily basis to load the patient claim and claim services data to hive tables.This patient claim and services data is further utilized in advanced analytics.

Bengaluru, Karnataka, India

Created notebooks with effective transformations and optimized timings using Pyspark with azure data bricks as IDE.∗ Spearheaded a big data project including requirement gathering, design, data collection, data modeling, data reporting,and unit testing and able to deliver it within the stipulated timeline.∗ Developed stored procedures and triggers in SSIS, T-SQL. Worked with structured and unstructured data, built pipelines,and collaborated with data scientists to build statistical models.∗ Deployed ADF pipelines to automate the ETL process using scheduled triggers that reduced manual intervention by 40%.Involved in developing the Logic Apps for email notifications and developing custom business transformations.∗ Orchestrated data integration pipelines in ADF using various activities like Get Metadata, Lookup, For Each, Wait,Execute Pipeline, Set Variable, Filter, until, etc.∗ Developed ADF Pipelines, that just needs a trigger, to transfer data from PROD to UAT and completes within fewminutes.∗ Collaboratively worked with business analysts to translate business insights into decisions and actions