Vineeth Kumar Email & Phone Number

Miami, Florida, United States

• Streamlined the Data Ingestion from different sources including Streaming, Batch, and Databases as part of AWS Data Lake implementation with the help of Spark on AWS Databricks, Delta Lake.
• Contributed to multiple ETL batch and Stream processing Spark jobs to transform and load the data to different data zones in Pearson Data Lake.
• Implemented the workflows using the Apache Airflow, and AWS Databricks Workflows to orchestrate job execution
• Integration of AWS Databricks Unity Data Catalog as part of Datalake Governance.
• Utilized Delta Lake, a reliable and scalable data lake solution on Azure Databricks, to ensure data integrity, ACID transactions, and versioning for large-scale data processing and analytics.
• Developed and orchestrated complex data workflows using Databricks Jobs and Workflows, enabling automated data processing, scheduling, and monitoring.

St Louis, Missouri, United States

• Built a framework for the Customer Master Data Platform to Profile the millions of customers and the framework enables to derivation of new custom attributes with different clients (Including Spark, SQL, and Python)
• Deployed Customer Master Data platform pipeline on AWS, all orchestrations done with lambda, step functions, and AWS cloud watch. Built the rest APIs using Python Django on top of our framework.
• Designed and Developed a Customer Unified User Profile framework (Discovering Audiences) to identify profiled users and Provide personalized offers. Deployed on EMR with 12 node cluster by caching terabytes of data.
• Worked on URL tracking with spark streaming and Predicted real-time results.
• Extensively Used Apache Spark, and Apache Flink for building real-time Streaming and batch jobs to process 10^6 events per minute. Contributed to multiple Streaming and structured streaming jobs which take care of.
• Designed and Developed data integration/engineering workflows on big data technologies and platforms (Hadoop, Spark, Hive).

New York, United States

• Worked extensively in AWS to Azure Cloud Migration on behalf of a Microsoft consultant.
• Migrated Click Stream pipeline from scratch to flow the data from transaction databases to data lakes and Dataware houses with the help of open-source selenium, Kafka, and spark stream processing
• Migrated Terrabytes of data from AWS to Azure by creating migration data pipelines in Azure Data Factory
• Had set up More than 150 Big data pipelines in Azure HD insights and orchestrated them through Airflow
• Experience in troubleshooting issues and failed jobs in the Hadoop cluster.
• SQL querying and performance tuning, creating backup tables.

Hyderabad, Telangana, India

• Built multiple rest endpoints for Finance Clients with Python Django framework that connects to MongoDB to fetch the required data.
• Written multiple Aggregated queries on top of MongoDB on Giga bytes of data.
• Built extract, transform, and load(ETL) process to migrate data from multiple types of data sources to destination Mongo DB.
• Built CI & CD deployment with Jenkins and Generated code quality report with Sonar Qube
• Extensively used Python language to develop web applications with MVC patterns.
• Identified issues with the database, and performed extensive optimizations to resolve them.