Shruthi S Email & Phone Number

Maryland Heights, Missouri, United States

• Expertise in designing and deployment of Hadoop cluster and different Big Data analytic tools including Pig, Hive, HBase, Oozie, Zookeeper, SQOOP, flume, Spark, Impala, Cassandra with Cloudera Distribution.• Installed Hadoop, Map Reduce, HDFS, AWS and developed multiple MapReduce jobs in PIG and Hive for data cleaning and pre-processing.• Implemented Partitioning, Dynamic Partitions, Bucketing in HIVE.• Created Hive tables with some partition column and stored in ORC format.• Created External tables in hive and stored the data.• Prepared Hive DDLs, queries and successfully executed on top of Spark.• Able to ingest the data onto HDFS and Hive from various platforms like Flat files and Mainframe files using Syncsort.• Created Syncsort task (dxt files) and also jobs (dxj files) with lot of conversions on the given fields.• Performed query optimization techniques using map side joins in hive. • Loading CSV files into data frame and performed Spark RDD transformations, actions to implement business analysis.• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.• Prepared Scala jar to process the data on top of Spark.• Involved in deploying the code through Source tree from Git to preprod, from preprod to prod.• Handle multiple issues during batch support.• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa. • Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.• Involved in validating the deployed files using data frames.• Participated in the PI meetings.• Participated in the Sprint Planning meetings.• Followed Agile Software development methodology, with bi-weekly sprints.• Participated in the Release/Demo meetings with the Client

Des Moines, Iowa, United States

• Analyse the technical requirements document to understand the requirements from customer.• Develop the new or maintain existing data analytics solution and solving the Adhoc request.• Involved in loading data into Spark RDD and do in-memory data computation and to generate the output response.• Involved in loading and transforming large sets of structured data.• Involved in extracting the data into data hub (BIDH).• Involved in converting the data into JSON data.• Involved in Migration of code from Dev to Production Environment.• Involved in Unit Testing, system integration testing for the Hadoop jobs and prepared test cases.• Converting the output into delimited file (csv, tab, pipe)• Compare the current month data(extracts created) with client historical data.• Imported data using Sqoop to load data from MySQL to HDFS regular basis.• Involved in Agile SDLC during the development of the project.• Implemented multiple MapReduce jobs in java for data cleaning and pre-processing.• Installed Oozie workflow engine to run multiple MapReduce, Hive and Pig jobs.

Wisconsin, United States

• Analyse the technical requirements document to understand the requirements from customer.• Develop the new or maintain existing data analytics solution and solving the Adhoc request.• Writing Sqoop jobs for moving data between Relational Databases and HDFS.• Experience in offloading historical and incremental data from RDBMSs sources to HDFS using Sqoop jobs.• Involved in optimizing the Hive queries.• Involved in writing Pig and Hive scripts.• Involved in migration of code from Dev to Production Environment.• Involved in Unit Testing, System Integration Testing for the Hadoop jobs and prepared test cases.• Converting the output of hive into delimited file (csv, tab, pipe)• Develop Map Reduce jobs in Java for analytics and data cleaning.• Import data from MySQL to HDFS, using Sqoop to load data.• Regularly tune performance of Hive and Pig queries to improve data processing and retrieving.• Created Hive tables to store the processed results in a tabular format.• Developed Sqoop scripts to load the data into hive tables.• Involved in ETL, Data Integration and Migration, Imported data using Sqoop to load data from Oracle to HDFS on regular basis.• Involved in gathering the requirements, designing, development and testing.• Perform big data processing using Hadoop, MapReduce, Sqoop, Oozie.• Writing CLI commands using HDFS.• Ensured NFS is configured for Name Node.• Moved all log/text files.• Written the Hive queries to process the data as per the requirement and loaded the data into SQL and HIVE table for reporting.• Implemented partitioning and bucketing for the tables in Hive.• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.• Worked on migrating MapReduce programs into Spark transformations using Scala.• Developed MapReduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion on Hive schema for analysis.