Senior Data Engineer
As a Data engineer I was involved in building the data modelling and ETL frame works to analyze the customer care call center data using the reporting dashboard in Power BIModelling Reporting Framework in Azure Data Bricks/SQLModeling Azure Zen 2 data lake for storage of the dataWritten python scripts to automate the process of pulling the source data using API and salesforcesystems.Extensively worked on SQL tables design, loading data into SQL tables and data bricks tablesImplemented Spark SQL to transform the raw data into refined data layer.Setting up the Azure Data bricks Unity Catalog for data governance.Created the Reporting layer in Azure Data Bricks for the Power BI reporting needs.Scheduling the Jupiter notebooks using the Azure data factory (ADF) for daily runsDeploying the code using Git hub Actions