Director Of Data Engineering
Current• Transitioned from legacy systems to Azure cloud infrastructure, enhancing data processing speeds and reducing costs.• Instrumental in the firm's digital shift through the integration of Big Data techniques.• Developed an end-to-end Azure data pipeline using Logic Apps, Data Factory, Databricks, SQL Server, and Azure Synapse Analytics.• Managed data projects from inception to delivery, aligning with business goals.• Set up a company-wide data lake for enhanced access via Synapse Analytics.• Integrated real-time data processing via Databricks with Spark, boosting real-time analytics.• Designed advanced ETL processes using Azure Data Factory, Databricks SQL, and Python.• Pioneered Azure's integration with GPT-4, employing Vector databases and Langchain for data-centric models.• Partnered with Data Science, IT, and security teams to bolster analytics, compliance, and safety.• Oversaw Azure Resource group budgets, emphasizing cost-efficiency.• Hosted workshops on modern data engineering tools, maintaining team expertise.• Designed graph database solutions with Neo4j and optimized data extraction using Cypher queries.• Established monitoring for data pipelines, integrating with Azure metadata through a Shiny dashboard.]• Engaged with business leaders to align data strategies with company objectives.• Delivered quarterly data strategy insights to executives, highlighting progress and plans.