Azure Devops Engineer
Current• Design, build, and maintain scalable ETL pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics to support data ingestion, transformation, and storage.• Integrate data from various sources (structured and unstructured) into Azure data lakes or data warehouses, ensuring seamless data flow across different business units.• Implement and manage Azure-based storage solutions like Azure Data Lake Storage (ADLS) and Azure SQL Database, focusing on performance optimization and cost-efficiency.• Ensure data security by implementing Azure RBAC (Role-Based Access Control), encryption (Azure Key Vault), and compliance with industry regulations like GDPR and HIPAA.• Utilize Azure Stream Analytics and Azure Event Hubs for real-time data processing and analytics, providing business insights on-the-go. Monitor data workflows and optimize them for performance and cost efficiency using Azure Monitor, Log Analytics, and Azure Cost Management tools.• Work closely with data science teams to deploy machine learning models on Azure Machine Learning and integrate them into business operations using data.• Automate deployment processes using Azure DevOps, ensuring smooth code integration and continuous delivery for data solutions.• Develop and automate administrative tasks and processes within the Azure environment using PowerShell scripts, such as managing data pipelines, scheduling tasks, and automating infrastructure provisioning.• Work with Linux-based systems to deploy, monitor, and manage large-scale data processing environments in Azure. This includes configuring services like Apache Hadoop and Spark on Linux virtual machines or Azure Databricks.• Use PowerShell and Python to define infrastructure as code, automating the provisioning of data engineering resources on Azure, and managing deployments efficiently.