Worked on designing and implementing enterprise-scale data platforms using AWS (Redshift, S3, Glue), Azure (Data Lake, Synapse Analytics), and GCP (BigQuery), delivering scalable solutions for business-critical applications handling petabytes of data. Working in a Scrum team of 12, developing a scalable data platform from scratch using PySpark, SQL, PLSQL, Azure Databricks and Azure Data Factory. Developing ETL pipelines of 40GB+ of customer data (e.g. driving units looked for) from different sources (SAP, CRM & MS SQL server) to find customer buying patterns. Led ETL tasks, overseeing data integrity and maintaining pipeline stability throughout the process. Implementing the GDPR attributes anonymization service using Python and SQL. Designed the Unit and Integration Testing for pipeline deployments with a code coverage 90% using SonarQube.

Developed ETL pipelines of 15GB+ customer data (e.g. display & search ads) to get better quality leads and conversion rates. Created various data Pipelines/Integration Runtime/Linked services using Azure Data Factory to process data from On- Premise to the AzureGen2Lake. Used PySpark and Spark SQL to develop applications for extracting, transforming, and analysing data from various formats, revealing insights into customer patterns. Architect and implement ETL and data movement solutions using Azure Data Factory, SSIS. Designed one-time load strategy for moving large databases to Azure SQL DWH. Extract Transform and Load data from Sources Systems to Azure Data Storage services using Azure Data Factory and HDInsight.

Designed and implemented robust database and reporting systems, aligning with business requirements and prioritizing integration, quality, and performance. Significantly enhanced query performance through expert analysis and optimization recommendations. Reviewed and crafted intricate queries and database objects to maintain a high standard of code quality. Used Agile Methodology of Data Warehouse development using Kanbanize. Developed data pipeline using Spark, Hive and HBase to ingest customer behavioral data and financial histories into Hadoop cluster for analysis. Working Experience onAzure Databrickscloud to organizing the data into notebooks and making it easy to visualize data using dashboards. Performed ETL on data from different source systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in InAzure Databricks.

Developed and created PL/SQL Packages, Procedures and Functions effectively by incorporate business logic as per the requirements. BI team enriched the analysis with reports like Monthly, Quarterly, Yearly, and Trend Reports to find out business strategies or gaps. Improved query performance through analysis of Execution Plans to optimize database tuning. Expertise in Client - Server application development using Oracle 11g/10g/9i/8i, PL/SQL, SQL *PLUS, TOAD, SQL*LOADER, (exp, imp) and impdp expdp.

Developed the data platform using SQL, PLSQL, ETL, Oracle and Dimensional modelling. Written SQL scripts to create objects like Tables, Views and Synonyms. Constructed comprehensive Test Cases and Scripts for rigorous database testing. Involved with business for requirements gathering and understanding and working with team collaboratively. Involved in Interaction with teh client for Database requirements.