Software Engineer
CurrentProject: ML Platform is designed for data scientists and users aiming to effortlessly create and manage ML models. It automates the entire model lifecycle management within the company. Key features include:• Model builds, ML pipelines, feature stores, dataset generation.• Both online and offline predictions (up to 10k RPM for online serving)• Autoscaling to dynamically optimize computational resources based on load demands• API for seamless integration with popular LLMs from different vendors, complete with cost tracking.The platform is built on a microservices architecture and leverages AWS computational resources.Responsibilities: Engaged in all kinds of backend and MLOps initiatives. Orchestrate cross-team collaboration to design and launch new services.Achievements:• Autoscaling for Online ML Models: Implemented an autoscaling system for online ML models, optimizing performance within budget constraints.• Batch Predictor Service: Led the cross-team development of a new batch prediction service, reducing the batch transform error rate from 15% to 3%.• Image Predictor Service: Launched a new service that reduced image prediction latency by 20% over the previous approach.