Prabhat Tripathi Email & Phone Number

San Jose, Ca, Us

1. Engaged with diverse datasets beyond payment metrics, including shipping and orders, while emphasizing a specific focus on payment metrics such as debit card and credit card transactions.2. Managed and implemented a comprehensive Regression Testing Framework, proficiently minimizing potential risks by maintaining consistent functionality of pre-existing code during the integration of new code or subsequent modifications3. Authored an efficient SQL script tasked with determining the total slot milliseconds and total bytes processed when using the COALESCE function across different job IDs. This was particularly useful when executing two distinct SQL scripts, each incorporating the COALESCE function in a unique manner, thus enabling optimized performance measurement and enhanced script execution.4. Responsible for facilitating load data pipelines and benchmarking the developed product with the set performance standards.

Monroe, Louisiana , Us

1. Implemented Databricks notebook that reads raw data in parquet format and writes the data into Azure Blob Storage container in delta format, which allows for upsert operations on delta tables created in Databricks, for the incremental data ingestion2. Worked on importing data using Sqoop from RDBMS to HDFS and copy data from HDFS to Azure Blob storage by using Distcp (distributed copy) tool.3. Optimized SQL query using Pyspark which fetched the required results faster, in a few minutes compared to SQL.

Oakland, California, Us

1. Worked on large sets of structured and semi-structured historical data2. Involved in working with Sqoop to import the data from RDBMS to Hive3. Created partitioned and bucketed Hive tables to load the Data.4. Involved in cleansing and transforming the data.5. Used spark SQL to perform sort, join and filter the data.6. Performed data Aggregation operations using Spark SQL queries7. Compared the sensor data to a persisted table on a 24hr period to check if the machine is operating at optimal conditions and Used Kafka as a messaging system to notify the producer of that data and the maintenance department in case a maintenance is required

Madison, Wi, Us

1. Used Sqoop to fetch data from RDBMS.2. Ingested and transformed the data using spark SQL.3. Created partitioned and bucketed Hive tables to load the Data.4. Exported Spark SQL Data frames into hive tables stored as ORC Files.5. Ingested real-time logs from various Kafka producers.6. Used spark streaming to subscribe to desired topics(drug) for real time processing.7. Performed data Cleansing to meet business requirements.8. Responsible for performing sort, join, aggregations, filter, and other transformations on the data.