Senior Data Analyst - Mle
CurrentDeveloping a WebApp Tool for People and Organization(HR) which is to be deployed on Azure Web Services. This Dashboard tracks information of Headcounts, Joiners, Movers and Compensation of all Associates.My Role on the project is Data Engineer,My Responsibilities on the project are, 1. Understand the data sources that are required to find the desired KPIs 2. Prepare the Solution Architecture for the Project on Data Engineering Aspect. 3. Cleaning, Aggregating, Processing and Harmonizing the code by storing data at different layers such as bronze, silver, Gold (Medallion Architecture in Databricks). 4. Push the Incremental data to SQL Database. 5. Create a Azure Data Factory Pipeline and Triggers to Orchestrate the Flow. 6. Maintain the code in repos in Azure DevOpsTech Stack : Databricks, Delta Tables, Unity Catalog, Azure Storage Containers, Azure Data Factory, Azure SQL, Logic Apps, Azure DevOps, Delta lake