 Worked on live 110 data nodes Hadoop production cluster running on CDH 5.10.11 Worked with highly structured data of PB in size. Developed complex HQL’s for generating the Risk reports for Basel regulatory. Involved in data extraction and data clean up from Source. Derived business attributes in hive reports based on the complex transformation logics given by BA. Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance. Partition altering, repairing the hive tables in Sit (lower environment) for the testing purpose. Duplicate data cleanup in hive tables for critical Business reports and developed the functions in Unix scripts for avoiding the permanent duplicate data Insert in hive tables. Excellent understanding of Joins and optimized the long running quires using sub queries and Map Joins in hive. Analyzed the SQL scripts and designed the solution to implement using Pyspark. Loaded the Parquet files from HDFS to hive tables using Pyspark. Involved in batch data transformation and processed the data for reporting purpose using PySpark. Involved in loading data from LINUX file system to HDFS and HDFS to hive. Handled data quality issue (missing, incorrect PnL/Risk data) and provided the permanent solution for the same. Worked on Extraction Framework (BEF) utilized to pull data from Teradata to Hadoop using Sqoop and Data quality checks for the critical data elements (CDE) defined by Biz and produce exception reports. Experience in importing the data using Sqoop between HDFS and relational databases. Autosys and Control -M jobs setup and monitoring. Excellent back tracing skills of any data quality issues till source. Tested raw data and executed performance scripts. Collaboration with multiple upstream and downstream or source systems to reduce the number of data quality issues/data mismatch issues.

Project Description:Riskstore is a part of risk management services having a various Risk data for entire Risk and Analytics portfolio. Barclayard Data Platform provides central data storage and analytic capabilities to all traded data.The data is stored in Hadoop Cluster.Traded Data will be stored in Hadoop distributed file system and processed using Hive and Pig scripts which intern includes getting the data from the Upstreams like Credit risk,market risk,etc., process the files to obtain the analysed information from all the data, this process works on the concept of IDS and VDS also associated with batch label and filter.Roles & Responsibilities: Worked on a live 110 data nodes Hadoop production cluster running on CDH 5.10.11 Worked with highly structured and semi structured data of PB in size. Extracted the data from MySql into HDFS using Sqoop & Autosys. Running Hive scripts for business users/analyst requirements to perform adhoc analysis. Collaboration with upstream and downstream teams to reduce the number of feeds and issues related to feeds. Solved performance issues in Hive scripts with understanding of Joins concept. Handled change requests and Problem requests. Worked as a Major Incident Manager(MIM) for any high impact incidents affecting business Delievering the batch status to Business,upstreams and downstreams. Prepare and Publish PIR to management teams Participate in Disaster Recovery planning and exercises Log investigation using Unix and Applying workaround for Known issues Investigate the feeds which having duplicate,missing, incorrect PnL/Risk data

Worked as a Barclays Application Support Analyst, the main areas of focus are:Counterparty Risk Trading Desk, Portfolio Management Desk and Collateral Optimization Unit. CRT desk being the main focus includes Barclays CRT risk Engine Seahawk, Pricing Application DTRE, Processing and Book Management CRT Services Suite, ICE, Portfolio Risk as well as COU Exchanges and Portfolio Management Application Suite. Working as main L2 support with the traders taking on complex Production issues and managing them to resolution using ServiceNow and JIRA. Looking after the overnight batch performance and investigations into SLA breaches and implementing efficiency measures. Infrastructure management including new application and technologies on boarding as well as decommissioning of old equipment. Performing and validating complex releases encompassing Linux, Windows, Mongo DB. Implementing Incident, Problem and Change management to conform to ITIL v3 standards Finding Monitoring solutions for crucial services and feeds to ensure SLA management and minimum downtime. Collaboration with upstream and downstream teams to reduce the number of feeds and issues related to feeds. Barclays Disaster Recovery test procedures and documentation management. Cyber Ark Passvault on-boarding and implementation for system accounts. Autosys Instance migration for entire CRT estate with no downtime. Shared L1 team on-boarding and management related to CRT, PM and COU.