Senior Data Engineer
Current• Designed, built, and maintained ETL/ELT using Azure Data Factory/Synapse to collect data from disparate data sources and loaded them into Azure SQL databases/Azure ADLS Gen2/Azure blob, reducing data ingestion time by 50%. Azure Logic Apps is used within the jobs to send out success or failure email notifications of the pipelines.• Reading and Writing Data from and to Various Azure Analytics Services and File Formats using Databricks. Executed data transformation and aggregation using PySpark and Scala thus enabling big data processing capability in the analytics department.• Developed five expansive and several other automated reports in Power BI to go from raw data to deeper insights with over $5M in annual revenue. It made the key KPIs available in almost real-time to the relevant teams.• Developed and executed complex SQL queries for extracting business insights and transforming the data using aggregation, attribute construction and integration to make it suitable for BI visualization, increasing decision-making efficiency by 30%.• Architected robust data models that supported business intelligence insights, driving a 25% lift in data-driven decision-making.• Led the design and development of a scalable Azure-based analytics platform, improving data processing speed by 40%.• Implemented advanced analytics such as life cycle analysis, cohort analysis to information growth strategies that brought in over $3M of revenue.• Maintain a petabyte of data spread across 5 clients in SSMS. Troubleshoot and fix problems in case of failures in automated jobs.• Expertise in creating views, materialized views and stored procedures in SQL and Python Data Frames for analytics projects.