Pranesh K. Email & Phone Number

• Migration of existing data pipelines from On-premise to AWS.• Creating Batch jobs in AWS.• Ingestion of Clickstream events and processing using Spark in EMR.• Requirement gathering from Business analysts and creation of ETL contractdocument with all stakeholders.• Creation of Python frameworks and identifying issues while moving to AWS.• Enhancing Chef scripts as per the requirement.• Perform QE testing and Peer Review for pipelines.• Monitoring AWS cluster health using Wavefront and creating alerts.

Tokyo, Jp

• Design spark RDD and data frame jobs using pySpark for time critical batch applications.• Hive query tuning.• Ingestion of xml and json files from Qualys (HTTP calls) and Cloudpassage (REST API calls) and load to Datawarehouse handling pagination.• Creation of Spark, Mapreduce job using Talend.

Overland Park, Kansas, Us

• Handle more than 25 TB of data per day coming from various devices in Towers.• Built and maintain scalable data pipelines using HBase, Sqoop, Spark, Oozie workflow and Hive.• Using Dstream api consumed Kafka messages and pushed to Hbase tables.• Data modelling in Hive, Hbase.• Design and optimize dataflow approach considering application use case and large data volume.• Ingestion of files from Isilon system, aggregate data in Hive for respective vendor and push aggregated data to Teradata system for downstream reporting.• Work with Hortonworks team regarding issues due to Memory intensive operation with input source files contains more than 3K columns. • Design Oozie workflow to automate complex processes, schedule using coordinator and create artifacts for same.• Using Sqoop, Unix scripting to import incremental data into HDFS from Sql Server.• Creation of python scripts for purging of HBase table.

Buffalo, New York, Us

• Lead Team in migration of Sql server to Teradata to generate CCAR reports (Comprehensive Capital Analysis and Review) from Core Banking.• Building financial data modelling using Teradata FSLDM and building data marts for specific subject area.• Development of Informatica workflows for new data pipelines.• Worked on address validator in Informatica data quality to ensure the right address in output file.• Design LLD, HLD, UAT documents, code review documents, production deployment documents.• Coordinate with onsite team in all phases and lead offshore team to deliver on business priorities.• Developing Teradata views, stored procedures based on the Business Analyst requirement.

Sydney, Nsw, Au

• Created and Managed more than 50 services in IBM Message queues.• Created XSLT scripts to process xml files in IBM Message queues and load to Datawarehouse.• Extensively used ELT instead of ETL for better performance and also converting most of the extraction logic to the view.• Understand the complex existing system and come up with the POC to process log files using Hadoop to build Reconciliation Framework which helps to understand the records getting dropped in each layer for every pipeline.• Developed ETL jobs using DataStage 8.5 and Autosys.• Develop workarounds to deal with bugs in ETL tools.• Developed complex views and procedures in Teradata and worked in various utilities like TPUMP, MLOAD, FASTLOAD, FASTEXPORT and bteq's.• Benchmarking each utility in Teradata with respect to the current system and load.• Created solution to prevent Teradata table locking by using Autosys variables.

Bachelor Of Electrical And Electronics Engineering, Electrical And Electronics Engineering

Education record