Azure Data Engineer
Current• Designed, developed, and maintained scalable solutions for data pipelines, ongoing metrics, reports, and dashboards to support management and executive analytical requirements and business needs. • Assisted in the seamless extraction, transformation, and loading (ETL) process, integrating data from diverse sources into Azure data storage solutions using Azure Data Factory• Initiated the implementation of complex big data processing solutions with Azure Databricks, overseeing extensive data exploration, preprocessing, and advanced analytics to drive actionable insights• Designed and implemented a scalable data warehouse solution on Snowflake by utilizing features like virtual warehouses, data sharing, and automatic scaling to optimize performance and reduce operational overhead• Developed and deployed data processing pipelines with PySpark and Python, facilitating streamlined analysis and transformation of extensive datasets• Utilized PySpark's distributed computing capabilities for tasks like data cleaning to ensure data accuracy and consistency, enhancing the quality of insights derived from the datasets• Implemented strong migration strategies, ensuring data integrity, consistency, and security throughout the transition and collaborated closely with cross-functional teams to identify and address potential challenges.• Orchestrated the development and deployment of modern analytics solutions using Azure Synapse Analytics, leveraging its integrated analytics services, including data warehousing and big data processing.• Played a key role in maintaining Azure HDInsight clusters, driving efficient data ingestion, cleansing, and transformation processes• Collaborated with cross-functional teams to implement advanced analytics solutions, Databricks' integration with Azure services such as Azure Data Lake Storage