Working on medellian architecture which involves ingesting data from different sources and applying business requirements and transformations, post which the data is loaded into raw, curated and semantic layer for Business Intelligence.Integrated Open AI in Databricks to build Gen AI solution by using GPT response for content generation and then loading it into target data stores. Worked on multiple aggregation, SQL queries, Stored Procedures and joins to cater business… Show more Working on medellian architecture which involves ingesting data from different sources and applying business requirements and transformations, post which the data is loaded into raw, curated and semantic layer for Business Intelligence.Integrated Open AI in Databricks to build Gen AI solution by using GPT response for content generation and then loading it into target data stores. Worked on multiple aggregation, SQL queries, Stored Procedures and joins to cater business needs.Worked on automating and generating Power Bi report from databricks by calling Rest API and then using an existing template to extract final pptx file.Worked on email notifications to track down the job runs post completion. Show less

Working for the client which is the second most largest ecommerce company in UK ranging in different domains like Retail, Finance, Supply chain and other Business towers.Actively participate in client meeting to communicate and understand the business requirements/needs and then prepare a data mapping sheet(DMS) which is used as a reference for data migration further.Devloped and deployed ADF pipelines to migrate data from On Premises to Azure cloud and from Azure cloud to… Show more Working for the client which is the second most largest ecommerce company in UK ranging in different domains like Retail, Finance, Supply chain and other Business towers.Actively participate in client meeting to communicate and understand the business requirements/needs and then prepare a data mapping sheet(DMS) which is used as a reference for data migration further.Devloped and deployed ADF pipelines to migrate data from On Premises to Azure cloud and from Azure cloud to Azure cloud environment.Implemented Dataflow using ADF to perform complex transformations/business logics on PII Data.Created pipelines to copy data in different formats and also maintained the record in the audit table using ADF and SSMSCreated Stored Procedures in Azure SQL to perform the requested DB activities.Worked on ADF pipeline orchestration, generalization and optimization.Technologies used: ADB, Pyspark,ADF, AZURE SQL, SSMS, PYTHON , SQL. Show less