Azure Data Engineer (4.5+ years)• Designed and developed scalable Azure Data Factory (ADF) pipelines for seamless data ingestion, transformation, and loading across diverse sources and targets.• Proficient in creating and optimizing DataFrames for large-scale data processing in Azure Databricks using PySpark.• Built robust data lake architectures leveraging Azure Data Lake Storage (ADLS) and implemented cost-effective partitioning strategies to improve query performance and storage efficiency.• Automated ETL workflows and orchestrated complex data processing tasks with Azure Synapse and ADF integration runtimes.• Implemented real-time analytics pipelines using Databricks Structured Streaming, enabling continuous data flow and analytics across distributed systems.• Collaborated closely with data scientists to curate data for AI/ML model training and ensure high-quality datasets for predictive analytics.