• Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.• Involved in complete Big Data flow of the application starting from data ingestion upstream to HDFS, processing the data in HDFS and analyzing the data and involved Low level design for MR, Hive, Impala, Shell scripts to process data.• Handling Hive queries using Spark SQL that integrate with Spark environment implemented in Scala.• Used Spark Streaming API with Kafka to build live dashboards; Worked on Transformations & actions in RDD, Spark Streaming, Pair RDD Operations, Check-pointing, and SBT.• Implemented POC to migrate map reduce jobs into Spark RDD transformation using Scala IDE for Eclipse• ETL Data Cleansing, Integration & Transformation using Pig: Managing data from disparate sources.• Exported analyzed data to the relational databases using Sqoop for visualization & Report generation• Data Warehousing: Designed a data warehouse using Hive, created and managed Hive tables in Hadoop• Workflow Management: Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig

• Involved in Cassandra and Teradata for Nosql database.• Involved in designing, installations and maintenance of KAFKA and Talend.• Worked on DB2 to extract data and transfer it to Hadoop.• Loaded data into the cluster from dynamically generated files using FLUME and from RDBMS using Sqoop• Expert in Informatica for data analytics, data integration and management.• Worked on Sequence and ORC files, bucketing, partitioning for Hive performance and storage improvement• Used Oozie scheduler to automate the pipeline workflow and orchestrate the map reduce jobs that Extract the data on a timely manner• Created Design documents, Architectural Documents and Technical documents for POC• Managed and reviewed Hadoop log files to identify issues when job fails

Project 1:• Developing parser and loader MR application to retrieve data from HDFS and store to HBase and Hive• Built ETL workflow to process sales and marketing data on hive tables• Importing the data from the MySql and Oracle into the HDFS using Sqoop• Experienced on loading and transforming of large sets of structured and unstructured data• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS• Created Hive Internal and External tables and loaded the data in to tables and query data using HQL• Used Hue for UI based PIG script execution, Oozie scheduling and creating tables in Hive• Deployment of applications using AWS EC2• Written Map Reduce java programs to analyze the log data for large-scale data sets• Participated in building CDH4 test cluster for implementing Kerberos authentication. Installing Cloudera manager and HueProject 2:• Work within the enterprise architecture team and designed system using SOA architecture• Worked with development team in analyzing data obtained from production systems and massaging them into readable format using MS SQL Server SSIS and Analytical Service (SSAS)• Designing Applications Architecture, HLD, LLD and Test plan and other architectural documents• Database designing and Written T-SQL, SQL queries, stored procedures, functions, triggers etc.• Client Management and Delivery Management under cost, quality and schedule.• Project Management activities like planning, resource allocations, tracking and people management• Created use case, class, package, sequence diagrams using MS Visio.• Performing User Acceptance test & Analyzing the Data and Defect fixing

• Application framework designing and development including database designing.• Designed and developed data pipeline using SQL Server 2010, SSIS, SSAS and related technologies.• Configuration Management, Code Review, Build Deployment Management and Releases.• Provided data modeling and database design using ER-Studio and Enterprise Architect.• Involved in various phases of Software Development Life Cycle (SDLC) of the application• Developed user interfaces using JSP, HTML, XML and JavaScript.• Used JavaScript for client side validation and created external style sheets using CSS.• Web application development using J2EE: JSP, Servlets, JDBC, Hibernate, JUnit and Apache Log4J, Web Services, Message Queue (MQ).• Involved in the preparing the Unit test plan documents• Support to setup process implementation for ISO 9001, ISMS 27001 process documentations.• Implemented security in modules as Identity, Authentication, and Authorization

• Application designing and development including database modeling.• Export or Import data from other data sources like flat files using Import/Export of DTS• Requirement Analysis, design, development and implementations.• Broadly involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using SSIS• Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.• Used ASP.Net web application in LINQ to SQL for database connectivity.• Developed the necessary Stored Procedures and created Complex Views using Joins for robust and fast retrieval of data in SQL Server using PL/SQL.• Experienced in designing report layouts in SSRS and deployed cubes using SSAS• Worked with Team Foundation Source (TFS) control which stores all code, as well as a record of all changes and current check-outs in SQL Server database.• Coordinated with technical team for production deployment of software applications for maintenance