Software Engineer
CurrentBuilding the recommender system that powers the Twitter Home Timeline, the largest scale and most impactful consumer product at Twitter (> 75% of time on Twitter is spent on Home).Experienced in building scalable and fault-tolerant distributed systems. Shipped end-to-end improvements to the entire online and offline codepath covering critical online serving infrastructure, feature engineering, performant data pipeline and running advanced A/B tests at scale.Optimized online serving path component with SLO of 10M predictions/sec, 50M QPS. Improved service tail latencies by > 20% and achieved massive reduction in ranking failures. Improved service scalability, reliability, and maintainability.Redesigned streaming and offline data pipelines to increase production training data collection by 30, O(100TB), while removing training/inference discrepancy. This resulted in significant model and ranking quality gains on Twitter’s largest consumer facing product surface: +16M UAM, +500k unique engagers.Migrated core service components out of a legacy codebase into a more performant framework.Keywords: distributed systems, backend, ML, Python, Java, Scala, AWS, GCP, BigQuery, SQL, ETL, Kafka, Hadoop