• Developed ETL pipelines on GCP using Apache Beam and Dataflow to process large-scale data in real-time, resulting in a 20% improvement in data processing time.• Built and deployed data pipelines using Cloud Composer and Cloud Functions, enabling seamless integration with other GCP services such as BigQuery, Pub/Sub, and Cloud Storage.• Implemented monitoring and alerting mechanisms using Stackdriver, enabling proactive issue identification and resolution in GCP data pipelines.• Designed and executed end-to-end testing strategies for GCP data pipelines, ensuring the accuracy and completeness of data from ingestion to analysis.• Utilized DevOps practices and tools such as Jenkins, Terraform, and Ansible to automate GCP infrastructure deployment and configuration, resulting in a 50% reduction in deployment time.• Worked with Python, SQL, and Bash scripts to develop custom data transformations and data quality rules, resulting in a 25% reduction in data processing errors.• Designing data models and schemas for Cosmos DB to accommodate the specific needs of the application and ensure optimal performance for different types of data.• Developed and maintained CI/CD pipelines on GCP using Cloud Build and Cloud Run, enabling seamless code deployment and testing in a controlled environment.• Implemented data versioning and lineage tracking using tools such as Data Catalog and Data Studio, enabling auditability and traceability of healthcare data in GCP.• Conducted capacity planning and scaling of GCP data pipelines using Kubernetes and Cloud Autoscaling, ensuring optimal performance and cost-efficiency.• Developed multi-cloud strategies in better using GCP (for its PAAS).• Designed and developed Spark jobs with Scala to implement end-to-end data pipelines for batch processing.• Developed data pipeline using Flume, Kafka, and Spark Stream to ingest data from their weblog server and apply the transformation.

• Designing and deploying AWS Solutions using EC2, S3, EBS, Elastic Load balancer (ELB), auto-scaling groups• Set up and build AWS infrastructure for various resources, VPC EC2, S3, IAM, EBS, and Security Group. Auto Scaling, and RDS in Cloud Formation JSON templates.• Designed AWS Cloud Formation templates to create custom-sized VPC, Subnets, and NAT to ensure successful deployment of Web applications and database templates.• Extracted data from a variety of sources such as MySQL, Oracle 11g, MS SQL server, etc., and analyzed large data sets using tools such as SQL, SAS.• Developed stored procedures in MS SQL to fetch the data from different servers using FTP and processed these files to update the tables.• Designing and implementing data ingestion pipelines to efficiently extract data from various sources such as databases, data lakes, APIs, and streaming platforms into Azure data storage solutions like Azure Data Lake, Azure Blob Storage, or Azure SQL Database.• Performed data analysis and profiling of source data to better understand the sources.• Work related to downloading Big Query data into pandas or Spark data frames for advanced ETL capabilities.• Carried out data transformation and cleansing using SQL queries, Python and Pyspark.• Wrote scripts in Hive SQL for creating complex tables with high-performance metrics like partitioning, clustering, and skewing.• Implementing data ingestion pipelines to load data from various sources into Cosmos DB, such as streaming data from Azure Event Hubs, Iot devices, or other databases.• Designing and implementing data models to extract, transform, and load (ETL) data from various sources into Azure Data Lake or Azure SQL Data Warehouse for use in Power BI.• Created ETL Pipeline using Spark and Hive for ingest data from multiple sources.• Was responsible for ETL and data validation using SQL Server Integration Services.

• Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data.• Using Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics, extract, transform, and load data from source systems to Azure Data Storage services. • Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.• Implemented Proof of concepts for SOAP & REST APIs and utilized REST APIs to retrieve analytics data from different data feeds.• uilding data transformation workflows within data pipelines using Azure Data Factory, Azure Databricks, or Azure Synapse Analytics to cleanse, enrich, and transform raw data into a structured format suitable for analysis and reporting.• Developing interactive and visually appealing dashboards, reports, and data visualizations in Power BI to enable data-driven decision-making.• Identifying and resolving performance bottlenecks in Cosmos DB through indexing strategies, partitioning techniques, and query optimization.• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool, and backward.• Hands-on experience in developing SQL Scripts for automation purposes.• Developed Spark applications using PY-spark and Spark-SQL for data extraction, transformation, and aggregation from multiple le formats for analyzing & transforming the data to uncover insights into customer usage patterns.• Responsible for estimating the cluster size, monitoring, and troubleshooting the Spark data bricks cluster.• Experienced in performance tuning of Spark Applications for setting the right Batch Interval time, the correct level of Parallelism, and memory tuning.

• Creating mappings, sessions, and workflows to use with Informatica PowerCenter to load data for Ultimatix projects from source to target database.• Creating Informatica positions for SCD Types 1, 2, and 3. Transferring the processes and mappings from development to QA, QA to production, and unit testing the procedure.• Participated in the seamless migration of the mappings, sessions, workflows, and repositories from Informatica 9.0 to Informatica 9.6.• Implementing data integration processes using T-SQL and tools like Azure Data Factory to orchestrate ETL workflows.• Created the Informatica Mappings by leveraging the Aggregator, SQL overrides in Lookups, source qualifiers, and Router to govern data flow into different targets.• Developed shell scripts to optimize the Informatica workflows' ETL flow.• Created Sessions, gathered data from multiple sources, processed it as needed, and loaded it into the data warehouse.• Designing and implementing relational database schemas using T-SQL in Azure SQL Database or Azure SQL Data Warehouse.• With the Informatica Power Center Designer, strong mappings were created using a variety of transformations, including Filter, Expression, Sequence Generator, Update Strategy, Joiner, Router, and Aggregator.• PL/SQL stored procedures were created, and push-down optimization tuning was used to minimize execution times by 30%.• Created materialized views, planned jobs, ETL workflows, and reporting that allowed data to move between eight ERP systems.• Identifying and resolving performance bottlenecks in T-SQL queries through query optimization techniques such as indexing, query plan analysis, and rewriting complex queries for better performance.• Setting up and managing Cosmos DB multi-region replication to ensure data availability and low-latency access for globally distributed users.

• Developed and examined business needs to create technically proficient data solutions that can be put into practice.• Analyzing classified data items for data profiling and mapping from source to target data environments and created working documents to back up results and assign particular responsibilities.• Used complicated SQL to analyze and profile data from a variety of sources, including Teradata and Oracle.• Created, and executed a SQL script to carry out the development of views, store procedures, and indexes.• Participated in meetings for gathering information and JAD sessions to deliver a business requirements document and a draft logical data model.• Specify how the data will be sourced from sources and loaded into DWH tables by defining the ETL mapping specification and designing the ETL procedure.• Created mappings using the transformations Source Qualifier, Expression, Filter, Lookup, Update Strategy, Sorter, Joiner, Normalizer, and Router.• Carrying out data administration tasks and completing ad-hoc requests in accordance with user requirements using data management software and tools like Perl, Toad, MS Access, Excel, and SQL.• Recognized and examined sources of data from flat files, Oracle, and SQL Server.• Used ERWIN to perform forward and reverse engineering and apply DDLs to databases to restructure the current data model.• Created ETL specification documents to load the data into the target utilizing different transformations in accordance with the business needs.