Senior Manager-Data Enginerring
CurrentProven Record in Enhancing Data Quality• Identified limitations of unit/sample testing, addressing ongoing production issues.• Developed an automation tool using PySpark - Quality Assurance Framework (QAF) covering data completeness, correctness, volume, schema, and exploratory data analysis.Problem Solving• Identified root causes of data quality issues using the 5 Whys technique.• Conducted workshops to demonstrate the impact of poor data quality on business outcomes.• Engaged key stakeholders in the decision-making process to increase buy-in.Automation and Monitoring• Integrated QAF framework into the data engineering pipeline for weekly data quality monitoring.• Created Power BI Data Observability reports for monitoring data health.Critical Failure Alerts & Automation• Developed alert test cases for critical model output failures.• Implemented a job to halt the DE pipeline if any alert test case fails.• Automated data monitoring, reducing data defect tracking (DDT) by 60%.Extending QA for Data Science• Enhanced QAF to validate data science output files, addressing missing forecasts, data loss, threshold checks, negative forecasts, and duplicates.• Integrated quality checks with the ML pipeline and set up email alerts based on results.Leadership & Best Practices• Established QA processes like QA-DEV-BA collaboration, early UAT testing, and shift left/right analysis, enabling early defect detection and reducing development effort.• Consulted on QA, aiding clients in advancing through quality maturity models.• Led scrum ceremonies and managed knowledge in Confluence as a Scrum Master.• Achieved near-zero attrition over six years through effective mentoring and high team engagement.Recognitions• Point of contact for BFS domain data testing practice in Bangalore and APAC.• Received Spot Award for 75% automation test coverage and Shining Star for a successful AML
