Kumar Vinay Email & Phone Number

Frisco, Tx, Us

Frisco, Texas, United States

• Designed and implemented Cloud Native system architecture, reducing latency by 20% and increasing data throughput by 25%, resulting in more efficient real-time insights for business decision-making.• Leveraged GCP services including Google Composer, Airflow, Big Query, and Bigtable to develop scalable data solutions.• Developed multi-cloud strategies in better using GCP (for its PAAS) and Azure (for its SAAS).• Developed complex Microsoft, composer flows for automating flows when using both clouds together and Expertise in Designing and deployment of Hadoop cluster and different Big data Analytics tools including Pig, Hive, Sqoop, Apache spark• Developed complex SQL queries using Hive/PySpark data frames, optimizing joins for processing extensive datasets, leading to a 30% improvement in performance.• Utilized Python/Scala programming to craft data pipelines, enhancing data processing efficiency by 15%.• Engineered and deployed distributed, reliable, and scalable data pipelines for real-time and unstructured data, driving timely insights.• Collaborated with cross-functional teams, contributing to project design, delivery estimates, and code reviews, resulting in streamlined development processes.• Translated business requirements into usable and scalable software solutions, aligning technology with strategic goals.

United States

• Worked on testing and validating data by developing pythn and between landing conformance and consumption layers for a unified platform to get the data from multiple sources into snowflake.• Successfully developed and executed test plans , test cases, and scripts to ensure the quality of the platform.• Performed root cause analysis and provided recommendations for fixing the issues identified in the testing process.• Collaborated with cross-functional teams to identify and resolve the issues that arose during the testing phase.• Demonstrated experience in working on TACO automation testing.• Skills: Python, SQL, Snowflake, AWS.

United States

• Experience in building multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation in GCP and coordinate task among the team.• Instrumental in scaling the Sylog Next gen [Next generation] platform for processing ~13 Million events per day.• Sylog Next generation is highly available and highly scalable event driven and real time distributed data pipeline• Built a streaming data analytics platform to process ~500Tb volume events to process telemetry data on ~80Bs of raw data per day.• Design and implement various layer of Data Lake, Design star schema in Big Query.• Using g-cloud function with Python to load data into big query for on arrival csv files in GCS bucket.• Designed Pipelines with Apache Beam, KubeFlow, Dataflow and orchestrated jobs into GCP.• Developed and Demonstrated the POC, to migrate on-prem workload to Google CloudPlatform using GCS, Big Query, Cloud SQL and Cloud DataProc.• Documented the inventory of modules, infrastructure, storage, components of existing On- Prem data warehouse for analysis and identifying the suitable technologies/strategies required for Google Cloud Migration.• Design, development, and implementation of performing ETL pipelines using python API(pySpark) of Apache Spark.• Built transformation layer using DBT tool for big query.• Exposure on IAM roles in GCP.• Create firewall rules to access Google data procs from other machines.• Process and load bound and unbound Data from Google pub/subtopic to big query usingcloud Dataflow with Python.• Setup GCP Firewall rules to ingress or egress traffic to and from the VM's instances based on specified configuration and used GCP cloud CDN (content delivery network) to deliver content from GCP cache locations drastically improving user experience and latency.• Skills: GCP, Airflow, Cloud SQL, Big Query, Cloud Data Proc, GCS, Cloud SQL, CloudComposer, Hadoop, Hive, Spark, Python, SQL Server, AZURE Data factory, CICD devo’s.

United States

• Developed stored procedures, functions, views, and triggers in PL/SQL to be used in the ETL process.• Utilized Sqoop, Flume and Hadoop File System API’s for implementing data ingestion pipelines.• Created Hive tables, loaded with data, and wrote Hive queries to process the data.• Created Partitions and used Bucketing on Hive tables and used required parameters to improve performance and developed Hive UDFs as per business use-cases.• Developed Hive scripts for source data validation and transformation. • Automated data loading into HDFS and Hive for pre-processing the data using Oozie.• Collaborated in data modeling, data mining, Machine Learning methodologies, advanced data processing, ETL optimization.• Enforced referential integrity in the OLTP data model for the consistent relationship between tables and efficient database design.• Used Pentaho Import Export utility to Migrate Pentaho Transformations and Job from one environment to others (DEV/QA/PREPROD/PROD).• Worked with Business users and Business analysts to understand and analyze ETL and reporting requirements.• Worked with homogenous and heterogeneous sources and targets like relational tables, delimited and fixed width flat files and CSV format files while creating mappings.• Worked on various data formats like Avro, Sequence File, JSON, Map File, Parquet, and XML.• Experienced in working with Hadoop from Cloudera Data Platform and running services through Cloudera manager.• Experienced in Hadoop Production support tasks by analyzing the Application and cluster