· Primarily responsible for creating Spark applications using Scala to process huge amounts of data.· Requirement gathering and translating them into detailed architectures and designs.· Responsible for building scalable distributed data solutions using Hadoop· Responsible to manage the data coming from different sources, processing and sharing with the downstream· Facilitating data migration, setting up the strategy and implementing it.· Responsible for enhancements and troubleshooting issues during testing.· Defining data migration strategy to migrate history data from Mainframe to Hadoop using Apache Sqoop· Export processed data to the relational databases using Sqoop for visualization and to generate reports for the BI team

· Design and development of Spark applications using Scala to process Huge amount of zipped JSON /text/CSV data· Created Hive tables to load large sets of structured, semi-structured and unstructured data coming from a variety of portfolios and processed data using Spark· Supported code/design analysis, strategy development and project planning.· Created reports for the BI team using Sqoop to export data into HDFS and Hive.· Participating in Iteration planning and signing-up user stories

· Analyzing the client requirements· Creation of technical design document· Testing of enhanced programs with prepared unit test plans and documenting test results· Testing enhancements in test regions with the prepared System test cases and User acceptance test cases.· Resolving technical and functional queries of team members.· Test plan preparation, debugging code and code reviews for peer.· Preparation and updating knowledge repository documents for application· Client communication and status reporting.

· Coding/enhancing the application· Resolving production tickets of Critical/High/Medium/Low priority· Assignment of production tickets/tasks to team members as per priority and tracking progress as lead· Talking to Data stage team and Identifying the delays in execution of Interface Jobs/delay of Interface files· Leading and influencing technical conversations with other support teams.· Documenting resolutions of the priority tickets.· Scheduling and organizing regular team meetings· Providing KT to the new team members.· Preparing weekly, monthly and quarterly reports of tickets closed vs assigned and sharing with offshore/onshore management.