Managed the development and execution of ETL data pipelines in Azure Data Factory, improving the processing of millions of financial transactions daily and facilitating faster financial reporting. Administered Azure Data Lake and Azure SQL Database for optimal data storage solutions, preserving customer data for over 10 million accounts and ensuring high availability. Leveraged Azure Databricks and HDInsight for Big Data processing, enhancing credit risk modelling and customer segmentation, and driving actionable insights for data-based decision making. Implemented robust data security measures using Azure Security Center and Azure Key Vault, protecting sensitive financial information, and ensuring compliance with GDPR and other financial industry regulations. Orchestrated automated loan approval workflows using Azure Logic Apps, reducing processing time and enhancing customer experience. Worked with Azure DevOps to streamline CI/CD pipelines, enabling faster updates to customer-facing banking applications and reducing time-to-market for software releases. Streamlined deployment and scalability of data processing environments using Docker containerization and Azure. Experience in Developing ETL solutions using Spark SQL in Azure Databricks for data extraction, transformation and aggregation from multiple file formats and data sources. Creating pipelines, data flows and complex data transformations and manipulations using ADF and Pyspark with Data bricks. High end supported (L3/Production issues) for already rolled out countries. Implemented real-time data processing using Spark Streaming and PySpark, enabling rapid insights from transaction data and improving fraud detection and risk management. Enhanced data quality and integrity by ingesting and transforming data from diverse sources, and developed ETL processes using SSIS and SSRS packages to facilitate data-driven decision-making. Played a pivotal role in cross-functional teams to develop data models and solutions that drove a 20% improvement in meeting business needs. Implemented robust data governance and security measures, safeguarding sensitive customer data, and ensuring compliance with banking industry regulations. Boosted system efficiency by optimizing data pipelines and queries, and improved speed of generating banking reports using efficient Hive queries. Enhanced data processing efficiency by working extensively with RDDs, Data frames, and Hive for data analysis and processing, and optimized batch processing of streaming data with Spark Streaming.