Key Achievements :-- Designed and implemented large scale complex data processing pipeline including three crucial modules of the product namely Data Quality, Data Transformation, Data Reconciliation which aligned with architectural principles and business needs.-- Worked on performance optimization, enhancement and testing of above components: 1) Addressed challenges related to data processing efficiency, achieving a 32% performance optimization throught the pipeline.… Show more Key Achievements :-- Designed and implemented large scale complex data processing pipeline including three crucial modules of the product namely Data Quality, Data Transformation, Data Reconciliation which aligned with architectural principles and business needs.-- Worked on performance optimization, enhancement and testing of above components: 1) Addressed challenges related to data processing efficiency, achieving a 32% performance optimization throught the pipeline. 2) Achieved a remarkable 43% improvement in storage efficiency in HDFS and Azure Delta Lake. 3) Achieved notable 37% reduction in runtime of the pipeline.-- Understood business requirement and designed, developed, tested solutions that adhered to big data architectural guideline and business requirements, some of the solutions included: 1) Enhanced the Data Transformation module with transpose functionality. 2) Maintained integrity of data processed by the pipeline and upgraded the pipeline to be fault tolerant without affecting the storage efficiency.-- Identified, designed, and implemented internal process improvements and automated manual processes.-- Collaborated with cross-functional teams, including Business Analysts, Clients, and QA teams, to identify and document high-impact business problems and proposed effective solutions without affecting client deployments.-- Debugged code in case of issues by investigating log4g log file.-- Documented detailed applications, specifications, translated technical requirements into programmed application modules by creating Higher Level, Lower Level, Deployement and Release Documents Show less