Data Engineer
Current• Act as architect for data platform within Azure• Transition data platform from Azure West 2 to Azure West 3 • Design, code, and implement streamlined DevOps architecture for Databricks Notebook deployment and Azure Data Factory deployment• Design, code, and implement approach of Qlik Replicate Streaming into Azure Event Hubs• Design, code, and implement near real time config-based Delta Live Pipelines to ingest and CDC Event Hub streaming data • Prove value of new utilities,tools, and methodologies• Optimize Databricks Clusters based on cost to performance ratio• Design, code, and implement external Hive metastore for Databricks and Synapse integration• Implement and design security model around Azure Databricks with integration in AD groups• Design and architect highly scalable ingestion framework that maximizes dollars saved per process