Srinadh Maddi Email & Phone Number

San Jose, Ca

Responsibilities:• Developing custom aggregate functions using Spark SQL and performed interactive querying.• CreatingHive tables, dynamic partitions, buckets for sampling and worked on them using HiveQL.• Using Hive tables and Hive SerDe’s to store data in tabular format.• Experienced with optimizing techniques to get better performance from Hive queries.• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further… Show more Responsibilities:• Developing custom aggregate functions using Spark SQL and performed interactive querying.• CreatingHive tables, dynamic partitions, buckets for sampling and worked on them using HiveQL.• Using Hive tables and Hive SerDe’s to store data in tabular format.• Experienced with optimizing techniques to get better performance from Hive queries.• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.• Worked on ETLInformatica for parsing the data, and then the parsed data is loaded to HDFS.• Working with NoSQL databases like Hbase to create Hbase tables to load large sets of semi structured data from various sources.• Debugging/Troubleshoot issues on UDF’sin Hive.• Managing and scheduling jobs on a Hadoop cluster using Oozie work flow.• Importing of data from various sources, performing transformations using Pig and loaded data into HDFS and extracted data from MySQL to HDFS using Sqoop.• Continuous monitoring and managing the Hadoop cluster using Hortonworks.• Worked on Oozie to automate data loading into HDFS and Pig to pre-process the data.• Experienced with different kind of compression techniques to save data and optimize data transfer over networks using Lzo, Snappy etc.,• Worked on installing cluster, commissioning and decommissioning of data node, name node high availability, capacity planning and slots configuration.• Worked on performance tuning forPigqueries.• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.• Developed the Pig UDF's to pre-process the data for analysis.• Involved in loading data from LINUX and UNIX file system to HDFS.Environment: Hadoop, HDFS, Hive, Pig, Flume, Sqoop, Spark, MapReduce, Cloudera, Avro, Snappy, Zookeeper, CDH, NoSQL, HBase, Java (JDK 1.6), Eclipse, MySQL and Ubuntu. Show less

Rochester, Mn

Responsibilities:• Worked on writing transformer/mapping Map-Reduce pipelines using Java.• Handling structured and unstructured data and applying ETL processes. • Collected the logs data from web servers and integrated in to HDFS using Flume.• Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run Map Reduce jobs in the backend.• Involved in loading data into HBase using HBase Shell, HBase Client API, Pig and… Show more Responsibilities:• Worked on writing transformer/mapping Map-Reduce pipelines using Java.• Handling structured and unstructured data and applying ETL processes. • Collected the logs data from web servers and integrated in to HDFS using Flume.• Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run Map Reduce jobs in the backend.• Involved in loading data into HBase using HBase Shell, HBase Client API, Pig and Sqoop.• Designed and implemented Incremental Imports into Hive tables.• Worked in Loading and transforming large sets of structured, semi structured and unstructured data.• Extensively used Pig for data cleansing. • Involved in collecting, aggregating and moving data from servers to HDFS using Apache Flume.• Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data.• Worked extensively with Sqoop for importing and exporting the data from HDFS to Relational Database system and vice-versa. Loading data into HDFS. • Involved in creating Hive tables, loading with data and writing hive queries that will run internally in map reduce way.• Experienced in managing and reviewing the Hadoop log files.• Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing the data onto HDFS. • Implemented the workflows using Apache Oozie framework to automate tasks.• Worked with Avro Data Serialization system to work with JSON data formats.• Worked on different file formats like Sequence files, XML files and Map files using Map Reduce Programs.• Involved in Setup and benchmark of Hadoop /HBase clusters for internal use.Environment: Hadoop, Big Data, HDFS, Map Reduce, Sqoop, Oozie, Pig, Hive, Hbase, Flume, LINUX, Java, Eclipse, Cassandra, Hadoop Distribution of Cloudera., PL/SQL, Windows, UNIX Shell Scripting, and Eclipse Show less

Chicago, Il

Responsibilities:• Worked on analyzing, writing Hadoop MapReduce jobs using JavaAPI, Pig and Hive.• Responsible for building scalable distributed data solutions using Hadoop.• Involved in loading data from edge node to HDFS using shell scripting.• Worked on installing cluster, commissioning & decommissioning of DataNode, NameNode high availability, capacity planning, and slots configuration.• Created HBase tables to store variable data formats of PII data coming from… Show more Responsibilities:• Worked on analyzing, writing Hadoop MapReduce jobs using JavaAPI, Pig and Hive.• Responsible for building scalable distributed data solutions using Hadoop.• Involved in loading data from edge node to HDFS using shell scripting.• Worked on installing cluster, commissioning & decommissioning of DataNode, NameNode high availability, capacity planning, and slots configuration.• Created HBase tables to store variable data formats of PII data coming from different portfolios.• Exported the analysed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.• Worked with using different kind of compression techniques to save data and optimize data transfer over network using LZO, Snappy, etc.• Analyze large and critical datasets using Cloudera, HDFS, Hbase, MapReduce, Hive, Hive UDF, Pig, Sqoop, Zookeeper, & Spark.• Developed custom aggregate functions using Spark SQL and performed interactive querying. • Used Pig to store the data into HBase.• Creating Hive tables, dynamic partitions, buckets for sampling, and working on them using HiveQL.• Used Pig to parse the data and Store in Avro format.• Stored the data in tabular formats using Hive tables and Hive SerDes.• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.• Worked with NoSQL databases like Hbase in creating Hbase tables to load large sets of semi structured data coming from various sources.• Implemented a script to transmit information from Oracle to Hbase using Sqoop.• Worked on tuning the performance Pig queries.• Involved in writing the shell scripts for exporting log files to Hadoop cluster through automated process.• Installed Oozie workflow engine to run multiple Hive and pig jobs.Environment: Hadoop, HDFS, Pig, Sqoop, Spark, MapReduce, Cloudera, Snappy, Zookeeper, NoSQL, HBase, Shell Scripting, Ubuntu, Linux Red Hat. Show less

St. Louis, Mo

Responsibilities:• Responsible for building scalable distributed data solutions using Hadoop.• Written multiple Map Reduce programs in Java for Data Analysis.• Wrote Map Reduce job using Pig Latin and Java API.• Performed performance tuning and troubleshooting of Map Reduce jobs by analyzing and reviewing Hadoop log files.• Developed pig scripts for analyzing large data sets in the HDFS.• Collected the logs from the physical machines and the OpenStack controller and… Show more Responsibilities:• Responsible for building scalable distributed data solutions using Hadoop.• Written multiple Map Reduce programs in Java for Data Analysis.• Wrote Map Reduce job using Pig Latin and Java API.• Performed performance tuning and troubleshooting of Map Reduce jobs by analyzing and reviewing Hadoop log files.• Developed pig scripts for analyzing large data sets in the HDFS.• Collected the logs from the physical machines and the OpenStack controller and integrated into HDFS using Flume.• Designed and presented plan for POC on impala.• Experienced in migrating Hive QL into Impala to minimize query response time.• Knowledge on handling Hive queries using Spark SQL that integrate with Spark environment. • Implemented Avro and parquet data formats for apache Hive computations to handle custom business requirements.• Responsible for creating Hive tables, loading the structured data resulted from Map Reduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.• Implemented Daily jobs that automate parallel tasks of loading the data into HDFS using autosys and Oozie coordinator jobs.• Performed streaming of data into Apache ignite by setting up cache for efficient data analysis.• Responsible for performing extensive data validation using Hive.• Sqoop jobs, PIG and Hive scripts were created for data ingestion from relational databases to compare with historical data. • Responsible for cleansing the data from source systems using Ab Initio components such as Join, Dedup Sorted, De normalize , Normalize, Reformat, Filter-by-Expression, Rollup.Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, Sqoop, Flume ,Oozie, Java, Linux, Teradata, Zookeeper, autosys, Hbase, Cassandra, Apache ignite Show less

India

Responsibilities: • Involving in Analysis, Design, Implementation and Bug Fixing Activities.• Involving in Functional & Technical Specification documents review. • Created and configured domains in production, development and testing environments using configuration wizard.• Involved in creating and configuring the clusters in production environment and deploying the applications on clusters.• Deployed and tested the application using Tomcat web server. • Analysis of the… Show more Responsibilities: • Involving in Analysis, Design, Implementation and Bug Fixing Activities.• Involving in Functional & Technical Specification documents review. • Created and configured domains in production, development and testing environments using configuration wizard.• Involved in creating and configuring the clusters in production environment and deploying the applications on clusters.• Deployed and tested the application using Tomcat web server. • Analysis of the specifications provided by the clients.• Involved to Design of the Application.• Ability to understand Functional Requirements and Design Documents.• Developed Use Case Diagrams, Class Diagrams, Sequence Diagram, Data Flow Diagram• Coordinated with other functional consultants.• Web related development with JSP, AJAX, HTML, XML, XSLT, and CSS.• Create and enhance the stored procedures, PL/SQL, SQL for Oracle 9i RDBMS.• Designed and implemented a generic parser framework using SAX parser to parse XML documents which stores SQL.• Identified the required data to be pooled to Hadoop, and created required Sqoop scripts which were scheduled periodically to migrate data to Hadoop environment.• Provided further Maintenance and support, this involves working with the Client and solving their problems which include major Bug fixing.Environment: Java 1.4, Web logic Server 9.0, Oracle 10g, Web services Monitoring, Web Drive, UNIX/LINUX Hadoop, Hive, Web Logic Server, JavaScript, HTML, CSS, XML Show less

India

Responsibilities:• Designed the user interface and navigation for the whole web application using XHTML, XML.• Assist in data migration, testing, deployment and validation operations• Extensive use of Collection Framework features like Map, Object, List to retrieve the data from Web Service, Developed JavaScript for flash widgets, which were used for site.• Implemented MVC architecture using Struts2.• Design (High level) of system with user team & develop functional… Show more Responsibilities:• Designed the user interface and navigation for the whole web application using XHTML, XML.• Assist in data migration, testing, deployment and validation operations• Extensive use of Collection Framework features like Map, Object, List to retrieve the data from Web Service, Developed JavaScript for flash widgets, which were used for site.• Implemented MVC architecture using Struts2.• Design (High level) of system with user team & develop functional specifications. • Integrated Spring and Struts to inject dependencies into Struts Action classes• Used Spring Framework for developing the application and used JDBC to map to Oracle database.• Designed animated icons, vector images, banners, templates and presentations with the help of Macromedia Flash MX and ActionScript 2.0• Developed CSS classes to GUI web page controls and forms.• Built templates and pages using Web 2.0 standards (XHTML, CSS, and JavaScript).• Developed JavaScript scripts with DOM and JQueryEnvironments: Core Java, JSP, Servlets, HTML, PL/SQL, Eclipse, Linux/UNIX, Junit, Struts, EJB, Oracle Show less