* Designed and implemented scalable data ingestion pipelines using Azure Data Factory, ensuring efficient extraction from various sources, including SQL databases, CSV files, and REST APIs.* Leveraged Azure Databricks for optimizing Spark jobs and integrating diverse data sources, enhancing data processing pipelines and facilitating efficient data ingestion and transformation.* Orchestrated seamless data migration of 8B data from on-premises SFTP server to Azure Cloud using Azure Data Factory, ensuring smooth and secure transfer.* Engineered ETL pipelines for historical and incremental data transfer from Postgres SQL database to Snowflake, enhancing data integration solutions.* Proficient in Azure Data Factory monitoring, and collaboration with cross-functional teams to implement best practices, ensuring high availability and performance of Azure SQL Database.

* Implemented Azure Data Factory pipelines for end-to-end ETL processes, extracting, transforming, and loading data into Snowflake from diverse sources.* Hands-on experience with Azure Cloud Services, Synapse Analytics, SQL Azure, and collaboration with Analytics and BI teams to develop globally adopted reporting solutions.* Executed Proof of Concepts for SOAP & REST APIs, engaged with business users for requirements gathering, and optimized data pipelines and Spark jobs in Azure Databricks.* Worked on migrating databases, estimating cluster sizes, and troubleshooting Spark Databricks clusters, developing Spark code using Scala and Spark-SQL/Streaming for efficient data processing.

* Built and managed multi-node Hadoop clusters, configured Cloudera Manager, and developed MapReduce jobs for data cleaning and preprocessing using Python.* Utilized Sqoop extensively for importing data from RDMS sources into HDFS, implementing transformations, cleaning, and filtering using Hive and MapReduce.* Implemented Hadoop jobs to process terabytes of data, designed and implemented robust APIs, and improved database performance by 25% through indexing strategies.* Extensive experience in importing/exporting data between HDFS and RDBMS using Sqoop, installing and configuring Hive, and writing Hive UDFs.* Worked in an Agile environment, utilizing Git for version control, maintaining code repository, and collaborating with Rally for user stories and tasks tracking.

* Developed and deployed SSIS packages for ETL from OLTP and various sources to Data Warehouse, utilizing Lookup, Fuzzy Lookup, and Slowly Changing Dimension transformations.* Worked as an SSRS and SSAS developer, creating reports, building facts and dimension tables for efficient analysis, and acted as a DBA to monitor client databases on production servers.* Performed ETL operations using SSIS for data validation, extraction, transformation, and loading into data warehouses.* Migrated SQL Server 2005 to SQL Server 2008, designed visual report solutions and interactive dashboards using SSRS and other data visualization tools.* Developed complex ETL packages, created database objects, and designed multi-dimensional objects using SQL Server Analysis Services (SSAS) in the MS SQL Server 2016 environment.

* Extensive experience in ETL processes, working with Data Modelers and DBAs to build data models and table structures, actively participating in designing ETL job flows.* Utilized ETL tools, including SSIS, DTS, and DataStage, for data extraction, transformation, and loading from heterogeneous source systems like flat files, Excel, Oracle, and SQL Server.* Demonstrated expertise in dimensional data modeling, designing SSIS packages, stored procedures, and implementing best practices for optimal performance.* Worked in an Agile Scrum Methodology, involved in SSRS and SSAS development, monitoring client databases, and migrating SSIS packages from SQL Server 2005 to SSIS 2008.* Developed complex ETL packages for pulling data from Oracle, loading monthly data from Excel to SQL Server, and creating interactive dashboards using parameters and various data visualizations with SSRS and other tools.