Principal Data Engineer
CurrentLead engineer responsible for migrating on premise integrations, analytics and dashboards to Azure. We’re currently utilizing a hybrid environment consisting of Azure Data Lake storage/Azure SQL/various REST endpoints, custom python scripting and on premise MS SQL Servers (2016,2017,2019) to support integrations, reporting and Power BI adoption across the firm. Our goal is to migrate 95%+ of our integration infrastructure from custom python scripts to ADF pipelines utilizing data flows, azure SQL and numerous linked services/datasets.