• Extract, Transform, and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL) and processing the data in Azure Databricks.• Streamlined the data integration process between GCP and Tableau, reducing data extraction and transformation time by 45%, resulting in faster data availability for Tableau users.• Designed and deployed a comprehensive data governance framework on GCP, ensuring adherence to data quality, privacy, and compliance standards across the organization.

• Utilized Python, The ETL pipeline was developed and programmed to collect data from the RedShift data warehouse.• Worked on developing ELT Pipelines using Apache Spark, AWS Glue, and Python.• Performed end-to-end Architecture & implementation assessment of various AWS services Amazon EC2, Amazon S3, Amazon RDS, IAM, Amazon Elastic Load Balancing, Auto Scaling, Cloud Watch, SNS, SES, SQS, Lambda, and EMR.• Wrote Spark applications for Data validation, cleansing, transformations, and custom aggregations.

• Worked on converting the multiple SQL Server stored procedures into Hadoop using Spark SQL, Hive, Scala, and Java.• Developed PySpark and SparkSQL code to process the data in Apache Spark on Amazon EMR to perform the necessary transformations.• Performed extensive exploratory data analysis using Teradata to improve the dataset's quality and created Data Visualizations using Tableau.• Experienced in various Python libraries like Pandas and NumPy.• Load the data into HDFS from different Data sources like DB2 using Sqoop and load into Hive tables.