Data Extraction: Proficient in extracting data from SAP HANA and OData using Azure Data Factory, ensuring efficient integration and management.Azure Data Factory Pipelines: Developed and optimized robust pipelines, reducing interruptions and improving data flow efficiency.Automated Logging: Implemented SQL stored procedures for automated pipeline run logging, ensuring data integrity and operational monitoring.Proactive Notifications: Configured Azure Logic Apps for real-time failure notifications, enhancing system reliability and response times.RESTful APIs and Data Storage: Managed data integration projects, extracting from RESTful APIs to Azure Blob Storage, focusing on XML/JSON formats.Data Transformation: Utilized Azure Data Flow for complex JSON/XML flattening and data transformation processes.Security and Compliance: Established and maintained Azure Key Vaults, enhancing data security measures.Data Management and SCD: Applied Slowly Changing Dimensions (SCD) Type 1 logic for maintaining data accuracy and integrity.Automation and Parameterization: Created dynamic datasets and linked services to streamline data operations.T-SQL Expertise: Advanced SQL scripting skills for data manipulation and management.Performance Optimization: Led performance tuning and troubleshooting efforts, significantly improving system efficiency.Tools and Technologies:Azure DatabricksPythonAzure Data LakeData ExtractionPySparkAPI IntegrationETL ProcessesAzure Data FactoryAzure Logic AppsData Loading

Data Platform Architecture: Directed the creation of a comprehensive data platform, utilizing Azure technologies for structured data engineering and ETL processes.Technical Documentation and Process Improvement: Authored detailed technical documents and refined operational processes to boost data handling efficiency.Global Team Integration: Collaborated with international teams to synchronize data engineering efforts across multiple regions.Diverse Technology Utilization: Proficient use of Azure Data Factory, Databricks, CosmosDB, and Azure Synapse for varied data engineering tasks.

Migration of workloads and data to Azure;Building data pipelines with Azure Data Factory;Tools used:- Microsoft Azure- Azure Data Factory- Azure Databricks- Azure Synapse Pipeline- Teradata Database- ETL Development

Design and development of data ingestion processes using Synapse Pipelines (Azure Data Factory) and SSIS.Creation and maintenance of stored procedures.Analysis, architecture, and design data warehouse.Development and maintenance of Power BI Dashboards.

Main activities:- Build data ingestion pipelines in Azure Data Factory;- Extracting data from a wide range of sources (API, JSON, Flat files, RDBMS)- Transformation pipelines in Azure Data Bricks using Pyspark and writing the data into delta lake tables;- Working with SQL inside the Databricks and Databricks notebooks;- Code versioning using GIT repositories with Azure DevOps;- Documentation of processes using Wiki whitin Azure Devops;- CI/CD using Azure Devops as developer onlyTechnologies and Methodologies: Azure Data Factory, Azure Databricks, Delta Lake, GIT, Scrum, Azure Devops, Databricks, Azure Data Lake Storage, Pyspark, SQL.

Responsible for planning, executing, and monitoring projects. Managing an overall schedule for each project, stipulating deadline, ensuring that each step is delivered on time. Maintaining good communication and involvement with the operation team involved.