Harini P Email & Phone Number

New Brunswick, New Jersey, United States

Houston, Texas, United States

United States

United States

I designed HBase tables for managing large structured datasets and handled Hadoop log files. Utilized AWS Glue for data transformation and Sqoop for MySQL-HDFS data transfers. Developed Hive UDFs, managed Hive DDLs and HQLs, and utilized Pig for ETL and data analysis. Created scripts and batch jobs for scheduling tasks with various file formats, and employed Zookeeper for cluster coordination. Implemented MapReduce, Pig, and Hive for data analysis, constructed data pipelines using Flume, Sqoop, and Pig, and managed Sqoop for large dataset transfers and XML to Avro conversions.

India

I developed and optimized Pig scripts and Hive queries for data processing and transformation, and utilized Apache Flume to load real-time unstructured data into HDFS. I designed ETL solutions for data acquisition, transformation, cleaning, and efficient storage on HDFS, and facilitated data loading from MySQL to Hadoop clusters using Sqoop. My work with PySpark and Spark SQL involved data cleansing, creating DataFrames and RDDs, and developing Spark code using Scala and Spark Streaming for fast data processing. I applied machine learning algorithms, such as K-nearest Neighbors and random forest, using Spark MLib on HDFS data, and used Tableau to visualize data outcomes. Additionally, I conducted exploratory data analysis (EDA) to identify trends, patterns, and anomalies within large datasets.