Senior Software Engineer
Current1. Designed and led the delivery of Starfish Enrichment Platform - a streaming system that scales Minstral LLM generated product content across 2B+ items in Amazon catalog. On a high-level the design included (a) system to prioritize items for regeneration based on maximum forecasted uplift, (b) gather input data based on product scope and model requirements, (c) hosting LLM container in EKS and integrating with a proprietary prompt manager and (d) data writer protected by guardrails like text quality checker, hallucination detection, customer A/B experiments and selling partner feedback. System is scaled to stamp 4M items per day increasing average item data completeness by 4x (10 to ~38 attribute backfills). 2. Designed Catalog Enrichment Eligibility Evaluator (CE3). Atypical of traditional enrichers that typically get executed on every product listing, this system uses a combination of current KPIs (glance views, search impressions), customer feedback (review summaries) and expected forecast (based on customer A/B experiments) to detect products that is expected to gain maximum uplift with LLM based content regeneration, thereby maximizing the ROI of costly LLM invocations. This system has been used to identify 40M items across 2 marketplaces for content regeneration in Q1 2024.3. Designed and led implementation of Product Type Classification system that is used to classify all listings with new seller content in Amazon catalog, amounting to ~100M classifications per day. The breadth of work includes (a) plugging into Amazon’s new item listing pipeline, (b) optimizing model containers across different instance types to maximize TPS, GPU utilization and minimize cost per inference (AWS inferentia was winning compute choice), (c) reconciling inference results across 20 marketplaces and product variations and (d) support self-service customizable business guardrail.