Data Engineer
CurrentKey Responsibilities:• Develop complex PySpark code for high-profile clients, including major sports teams and leagues.• Utilize Azure services extensively, including Azure Logic Apps, Azure Data Factory (ADF), and Azure Databricks, Azure Devops to design and optimize scalable data solutions.• Architect and implement data pipelines that handle billions of records daily, transforming raw data into actionable insights delivered through a Power BI-driven front-end solution.• Leverage Databricks for advanced data processing, machine learning model deployment, and optimizing big data workflows, ensuring high performance and reliability.• Designed large-scale data warehouse systems utilizing the Kimball methodology to ensure efficient data storage, organization, and accessibility.Key Accomplishments:• Redesigned existing data pipelines using Azure and Databricks, leading to a 20% improvement in OTT data insights for enhanced viewer engagement strategies.• Enhanced attendance processing workflows with Databricks and ADF, achieving 100% data accuracy,contributing to more reliable and precise reporting.