Azure Data Engineer
CurrentAt SUNRUN, I am responsible for architecting a seamless Datawarehouse structure, utilizing Azure services like Data Factory, Databricks, and Python to ingest data into the data lake. I design efficient tables and manage data in Azure Synapse and Databricks, creating high-performance clusters and fault-tolerant architectures. Using Databricks Delta Lake, I handle ETL workflows, orchestrating pipelines through Data Factory, while maintaining data integrity, overseeing table schemas, and managing user access for over 1000 tables. My role includes monitoring Azure environments, ensuring report accuracy, and liaising with Microsoft support for infrastructure concerns.