Vamsi G Email and Phone Number

I worked on optimizing large datasets using techniques like partitions, Spark in-memory capabilities, and effective joins to enhance performance during data ingestion. I upgraded HDInsight code to Azure Databricks for improved cluster optimization, utilized Azure Data Factory for integration tasks, and implemented Spark streaming with Kafka and Azure Event Hubs for real-time data processing. I developed customer recognition and risk profiling using pattern-matching algorithms in Hive, storing results in HBase, and deployed a proof of concept on AWS. My expertise includes transforming data in Spark with Scala and Python, managing data lakes on Azure, and supporting encryption on AWS DynamoDB. In addition, I led architectural strategies for Big Data solutions, developed data models using Erwin, optimized performance in OLTP and Data Warehousing environments, and created automation scripts and data transformations for diverse analytical and reporting needs.

I worked on creating and managing Azure Data Factory, where I implemented policies and utilized Blob storage for backup and storage solutions on Azure. I developed data ingestion pipelines from web services into Azure SQL Database, optimizing data integration on the cloud. I worked with PyTorch to develop deep learning models, utilizing features like the dynamic computational graph for easier debugging. Additionally, I managed Teradata servers globally, handling monitoring, tuning, indexing, and incident management, and supported development teams through automated scripts within SLAs. I leveraged PostgreSQL for analytics, geospatial data handling, and as a backend for CMS applications, and worked on multi-cloud environments, creating strategies to better utilize GCP for PaaS and Azure for SaaS.

With a strong foundation in AWS and Azure cloud technologies, I have managed data storage with AWS S3, data processing with EMR, and virtual Linux instances using EC2. Leveraging AWS CloudWatch for log analysis and Lambda functions for job automation, I have designed and maintained end-to-end data workflows. On Azure, I developed scalable cloud and analytical solutions across platforms, deploying custom Hadoop applications, migrating data to Azure Synapse, and utilizing Azure Data Factory for data transformation pipelines. Skilled in Spark and Hive, I performed data cleansing and analysis, executed complex MapReduce and SparkSQL joins in Azure Databricks, and integrated data into HBase for reporting metrics on dashboards. My work with GitHub, APIs, and Sqoop has facilitated seamless data ingestion and transformations from multiple sources, supporting analytical insights via Tableau and Excel.

I developed Spark applications in Python for distributed data processing, loading high-volume files into PySpark DataFrames, and handling structured data using snowflake schemas. I coordinated with Data Modelers to design dimensional models and implemented snowflake schemas for optimized data warehousing. Additionally, I generated high-level and low-level design documents for source-to-target transformations, ensuring improved data structure through snowflake schemas. I created test scenarios, test plans, and executed tests within SLAs, focusing on snowflake schema designs for accuracy. My role included designing ETL workflows, SQL queries, and shell scripts to incorporate complex business rules, migrating data from multiple sources, and optimizing queries in collaboration with the DBA team for efficient data loading. I also utilized Cassandra to manage diverse data types, used PowerShell scripting for maintenance and configuration, and employed Apache Airflow and Delta Lakes to build ETL pipelines and maintain data integrity, leveraging snowflake schemas throughout for consistent, unified data representation.

Worked closely with Business Analysts and report developers to define source-to-target specifications for Data Warehouse tables, ensuring data alignment with business needs. Provided data exports, ad hoc reports, and critical month-end financial reports, resolving user-reported issues by identifying and addressing discrepancies in standard reports. Developed and maintained a data warehouse for the PSN project, integrating online transaction data and managing user accounts and workspaces in PowerCenter. Migrated ETL processes from Talend to Informatica, utilizing complex PL/SQL and SQL queries for data analysis, and created UNIX/Windows scripts for file transfers and task automation.