Consultant Cloud Data Engineer
Current• Developed data pipelines in Azure Data Factory (ADF) using Linked Services, Datasets, and Pipelines for Extract, Transform, and Load (ETL) processes from diverse sources such as APIs and Blob Storage.• Managed data ingestion into multiple Azure services including Azure Data Lake, Azure Storage, and Azure Data Warehouse (Azure DW), with data processing carried out in Azure Databricks.• Implemented and optimized Apache Spark jobs in Azure Databricks for large-scale data processing and analytics tasks.• Utilized Databricks notebooks for interactive data exploration, analysis, and visualization.• Utilized an Azure Function App developed in Python to extract data from APIs.• Demonstrated expertise in Data Modeling and Data Warehousing utilizing Azure Synapse.• Proficient in version control and CI/CD practices, maintaining application source code in GIT for seamless development and deployment workflows.