Model inference infrastructure for Anthropic models. Orchestrating and scaling k8s distributed ML inference workloads on GPUs.- Managing Bedrock runtime, model serving, and inference workloads (On-Demand Throughput, Provisioned Throughput). Providing a high-scale, high-throughput, high-performance, multi-tenant distributed model inference infrastructure. Handle substantial input and output token processing rates, ensuring low latency, high resilience and efficiency. Key features include auto-scaling to dynamically adjust resources, robust hardware failure handling, and optimized capacity efficiency for model hosting functionalities. Make it supports complex model routing, accommodates models with long context lengths of up to 200k tokens, and manages concurrent requests effectively. Additionally, the infrastructure supports multimodal models, including both text and image processing, while guaranteeing compliance with trust and safety standards.

Worked on: AWS cloud architecture, distributed systems, OLAP Databases and Architecture

Single Molecule Biophotonics Group under Prof. Fabian Pinaud

• Ranked top 5 in the country for Under-16 and top 8 for Under-14, Under-18• 2-time Gold Medalist of Tennis Punjab Olympics (2018 and 2019), and won multiple national tournaments.