Data Engineer
CurrentDesigned and managed scalable data pipelines using Azure services, including HDInsight, DataLake, and Data Factory, resulting in a 25% reduction in processing time.Integrated diverse data sources (on-premises MySQL, Blob Storage, Azure SQL DB) into Azure Synapse, enhancing data accessibility and analytics capabilities.Developed and automated ETL processes using Python and PySpark, streamlining workflows and automating 75% of data tasks.Created data ingestion pipelines on Azure HDInsight and integrated with Cosmos DB for real-time data processing and analysis.Optimized table architectures in Azure SQL Data Warehouse, improving query performance by 40%.Implemented robust monitoring and error-handling mechanisms to ensure data integrity and system reliability.