Data Engineer
Data Pipeline Development: Designed and implemented ETL/ELT pipelines using Azure Data Factory for seamless data integration from various sources into Azure Data Lake Storage.Data Transformation: Leveraged Azure Databricks and Spark to perform complex data transformations, facilitating efficient data processing and analytics.Data Warehousing: Built and managed data warehousing solutions using Azure Synapse Analytics, optimizing data storage and retrieval for business intelligence purposes.Database Management: Administered and optimized Azure SQL Database instances to ensure high performance, security, and availability.Data Integration: Integrated data from on-premises and cloud sources using Azure Data Factory’s linked services and integration runtimes.Data Quality and Governance: Implemented data quality checks and governance policies to maintain data integrity and compliance with industry standards.Performance Optimization: Conducted performance tuning and optimization of data workflows and SQL queries to enhance system efficiency.Stakeholder Collaboration: Collaborated with data analysts, data scientists, and business stakeholders to understand data requirements and deliver actionable insights.DevOps and CI/CD: Employed DevOps practices, including the use of continuous integration and continuous deployment (CI/CD) pipelines via Azure DevOps, to streamline data engineering workflows.Documentation: Maintained comprehensive documentation of data workflows, pipeline configurations, and data models to ensure knowledge sharing and continuity.System Monitoring and Troubleshooting: Monitored data pipelines and systems using Azure Monitor and Log Analytics, quickly identifying and resolving issues to ensure data reliability.