Data Engineer
Current• Have to support the UI/frontend of the application. So, as part of it I am writing stored procedure to save the files that are uploaded from the front end into the DB. Also created stored procedure using PL/SQL to send data requested by the UI in JSON format. • Created SSIS files to read the data from Legacy SQL Server 2013 to the Azure DB. AS part of creating the SSIS files, handled different transformations to match the datatypes of the legacy and Azure SQL DB. • Worked with JSON data, flat files and table data to create appropriate stored procedures to save data into DB. Saved data into tables as per SCD1 and SCD2 logic according to the requirement • Created procedures using PL/SQL to handle table truncation, deleting specific records from tables and rolling forward selected data to next year data. • Created indexing on tables, building key relationships between tables also developed database diagrams using MS Visio • Created dynamic procedures which are used by Azure functions to trigger the ADF pipelines to save data into the ADLK • Also worked as a PowerBI developer in which I have developed few graphs like line chart, Area charts, tables, matrixes in PowerBI Desktop Application. • Created Views in Azure Databricks in SQL language which are consumed by PowerBI application as source dataset • Created measures and derived columns using DAX and M queries to make sure the graphs are as Dynamic as possible. So, that the graphs can also as the source dataset advances to next year