Rakesh Ranjan Email & Phone Number

Hyderabad, Telangana, India

Hyderabad Area, India

Novartis AccountCurrently working in Platform Team to help build up an open, modular, secure, multi-tenant and industry compliant Cloud based scalable Data Storage, Management & Analytics Platform Framework. Worked extensively in development of (Metadata driven ETL Framework) external Python based wrapper code which would:a) Onboard the tenants (jobs information) into RDS Postgres DB tables (all job information into various tables) with support of some AWS services.b) Trigger the onboarded job (using Airflow) to make Databricks job API call and execute the Databricks notebooks.Worked on implementation of important Framework features.• Parameter driven framework - reduces the amount of code developed and helps reuse existing code.• Scheduling with Airflow • Job Restartability: re-start failed jobs from Airflow.• Ad-hoc loading and reloading specific files/tables/partitions.• Parallel Jobs execution.• Onboarding/Off Boarding made easy for tenants.• Provides low level control - Activate and deactivate individual applications.• Jobs Stats: Captures critical job details, like Start time, elapsed time, record counts, job status.• Wide range of source systems - supports sources like RDBMS, Stream, API's, File systems.• Email Notifications: Wrapper script has the ability to notify the users.Involved in Onboarding of 1st Use case to the Platform.Extensively worked on:• Writing parametrized Databricks notebook code for performing common ingestion using pyspark code.• Ingested 70 tables of 15 sources using Databricks notebooks to S3 Raw bucket layer partitioned by load date.• Reading multiple tables from S3 multiple layer (raw, unified, refinement layers), applying complex spark transformation business logic and ingest final dataframe into target S3 target bucket layer.

Hyderabad Area, India

BigData | Hadoop | Spark Developer.Worked for DBS Bank (Development Bank of Singapore)Involved in automation of migrating the Financial Risk data from Oracle Database to HDFS to get the proper and fast analytics on it by which they can improve their service. Migrating the Business Logic which has written in thousands of Oracle Stored Procedures to Spark Data Frames in optimized way. For existing data from Oracle dumped into Hive tables by using Sqoop and computed the different types of data using Spark Data Frames and done the automation of Kafka with Oozie to stream the future data continuously into HDFS/Hive to compute accordingly business logic by using Spark Data Frames.• Analyzing the Oracle stored procedures based on the business documents and entire flow.• Mapping the functional logic for each and every stored procedure defined.• Converting the Oracle stored procedure logic into Spark Data Frames using Java.• Implemented the generic tool for file level validation according to the business logic.• Execution and implementation of efforts like coding, unit testing and Code review.• Unit Testing the code using Junit.• Involved in Sonar and Emma code coverage for Junit.• Code quality checks using Jenkins and peer reviews in an Agile methodology.• Optimization of Spark Data Frame code and submit query.• Direct interaction with BA (business analysts) to understand the requirements of assigned user story. • Supported SIT, UAT and multiple PROD deployments. • Direct interaction with Client and onsite coordinator.

Hyderabad Area, India

BigData, Hadoop & Spark DeveloperKey Technologies: HDFS, MapReduce Programming, Hive, Pig, ZooKeeper, Sqoop, Flume, HBase, Oozie. Spark, Spark Sql, Spark Streaming, Kafka, MySql, Scala, Core Java, Python (Basic), R (Basic).• Work with admin team to Setup of cluster in system and support on Hadoop.• Support all business areas of Data Services with critical data analysis that helps team members to make profitable decisions as a forecast expert and business analyst and utilize tools for business optimization and analytics.• Implementation of new technologies, which could be used to solve problem statements.• Formation of centralized Data Lake, which is then used to fetch information by user using web services.• Developed various methodologies to De-normalize the RDBMS data using Hive Joins and with numerous Hive operations along with graphical database implementation.• End-to-end performance tuning of Hadoop clusters routines against very large data sets.• Provide support for troubleshooting of Map Reduce Jobs, Hive Jobs, Hbase and data logs.• Demonstrates up-to-date expertise in Hadoop and applies this to the development, execution, and improvement.• Handled importing of data from various data sources, performed transformations using Spark Core, SparkSQL, Spark streaming and loaded data into various target source Kafka, HDFS, HBase, MySql.• Creating Hive tables and working on them using Hive QL.• Utilized Agile Scrum Methodology to help manage and organize our team of developers with regular code review sessions. Designed and Developed Data Ingestion Tool using Spark:This tool is built to take different data format Text, CSV, JSON, XML, MySql residing on HDFS, MySql, Kafka and perform transformations based on business logic using RDDs, DataFrame and DStream and dump the output to target system HDFS, HBase, Kafka and MySql.This tool is built on both Spark Batch and Spark Streaming, so that this gets fitment into any of the client’s requirement.

Hyderabad / Noida , India

KEY TECHNOLOGIES: HDFS, MapReduce Programming, Pig, ZooKeeper, Sqoop, Flume, Hive, HBase, Oozie.ACHIEVEMENT: • Four times winner of STAR Award, 2 times winner of Sustaining Edge Award. • Winner of Golden Crown award for Q1 2012 as star performer.• Submitted Case Study Document for PHR project implementing ICD9 to ICD10 conversion which is stored in Organization Repository for Best Practice. • Written and Published two White Papers on “Race, Ethnicity & Language - Information for HealthCare Organizations” and “Health Insurance Exchanges - Information for HealthCare Organizations” to HCoP.KEY PROJECTS: HADOOP PROOF OF CONCEPT(POC’S)1) Project Name: Data Mining on Wikipedia Data sets:This project was done as poc (proof of concept) to evaluate and benchmark the Hadoop cluster on a 500GB of data on a 10 node Hadoop cluster. Various tuning technique were evaluated like Combiner, compression (LZO,GZIP,Snappy), reusing JVM, speculative execution etc to verify the performance. 2) Project Name: Social Media AnalysisThis POC was to evaluate the brand sentiment for one of the automobile industry for collecting the data from the social media sites like Twitter, FB, specific URLs and analysing the data and generating reports. 3) Project Name: TFIDF TFIDF project was to identify the importance of a word in each page.Mainframes:COBOL, DB2, CICS, VSAM, JCL, ENDEVOR, XPEDITER, OPC Scheduling, • Coding the new programs according to the requirements. Adding the new requirements in the existing programs.• Complete setup of scheduling of jobs in Test and Production environment keeping consideration of all dependencies using TWS OPC scheduler. • Peer review of codes and Unit Test Conditions of team members, as per the programming standards. • Creating Weekly Status Report (WSR) of entire Team and share weekly status with Onshore Director. • Deployment Coordinator for components/modules to be deployed in production for all projects executed within the team.

Bangalore

Mainframes:• Analyzing the functional requirement from the design document.• Preparing Technical spec. and then the Program spec.• Coding the new programs according to the requirements.• Adding the new requirements in the existing programs.• Staging the program in the CHANGEMAN and promoting it to the CICS development region.• Thorough Testing of the new functionality added to the program in the development region. • Then performing the system testing.

Bachelor'S Degree, Mechanical Engineering

Bachelor'S Degree, Mechanical Engineering, First Class With Distinction

B.E. Mechanical