Sunil S Email & Phone Number

• Involved in importing and exporting data between Hadoop Data Lake and Relational Systems like Oracle, MySQL using Sqoop.
• Involved in developing spark applications to perform ELT kind of operations on the data.
• Modified existing MapReduce jobs to Spark transformations and actions by utilizing Spark RDDs, Data frames, and Spark SQL API’s
• Create and maintain optimal data pipeline architecture in cloud Microsoft Azure using Data Factory and Azure Databricks
• Create an Architectural solutions that leverages the best Azure analytics tools to solve our specific need in Chevron use case
• Design and present technical solutions to end users in a way the is easy to understand and buy into

US

• Developed Spark Applications by using Python and Implemented Apache Spark data processing project to handle data from various RDBMS and Streaming sources.
• Installed Kafka on Hadoop Cluster and Configured producer and consumer in java to establish connection from source to HDFS with Popular hash tags.
• Worked with Spark for improving performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Spark MLlib, Data Frame, Pair RDD's, Spark YARN.
• Performed tuning of Spark Applications to set batch interval time and correct level of Parallelism and memory tuning.
• Used Spark Streaming APIs to perform transformations and actions on the fly for building common learner data model which gets the data from Kafka in real time and persist it to Cassandra.
• Scheduling Spark/Scala jobs using Oozie workflow in Hadoop Cluster and generated detailed design documentation for the source-to-target transformations.

• Involved in importing data from Microsoft SQLserver, MySQL, Teradata. into HDFS using Sqoop.
• Installed and configured Hadoop Map reduce, HDFS, developer multiple Map Reduce jobs in Java and Scala for data cleaning and preprocessing.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS.
• Used Hive to analyze the partitioned and bucked data to compute various metrics of reporting.
• Involved in creating Hive tables loading data, and writing queries that will run internally in MapReduce
• Involved in creating Hive External tables for HDFS data.