Yi Cheng Email & Phone Number

United States

San Francisco Bay Area

Software engineer working on open source software Ray (http://tinyurl.com/ye25xzrv) and LLM perf for Anyscale. Focus on Ray's reliability and scalability. A major Ray committer since 2021.Major contribution:- Speed up model loading by 10x. Loading LLama-2-70B within 30s from S3 to GPU.- Scale Ray cluster to more than 4k nodes from 200. Reduce network usage by 10x (3GB/s -> 300MB/s) and CPU utilization.- Lead the effort to make Ray Serve an HA platform end to end.- Help OpenAI adopt Ray as their training platform, including improving scalability, scheduling performance and multi-client support.Some other contributions:- Support Redis-less Ray cluster and build the clean foundation to support multi storage backend.- Build workflow engine on top of Ray (http://tinyurl.com/ray-workflow)- Lead the effort to provide light weight dependence management for Ray (http://tinyurl.com/ray-runtime-env)Talks at Ray Summit- Ray summit 2023: Ray Scalability Deep Dive: The Journey to Support 4,000 Nodes (https://www.youtube.com/watch?v=1lYTmch4AbY)- Ray summit 2022: Highly available architectures for online serving in Ray (https://www.youtube.com/watch?v=zjRPu-ykHHI)- Ray summit 2021: Ray Internals: Object Management with the Ownership Model (https://www.youtube.com/watch?v=1oSBxTayfJc)Publications- Loading Llama-2 70b 20x faster with Anyscale Endpoints (http://tinyurl.com/fast-model-loading)- ExoFlow: A Universal Workflow System for Exactly-Once DAGs (https://www.usenix.org/system/files/osdi23-zhuang.pdf)

Redwood City, California

Software Engineer working database engine, including storage, computation

Menlo Park

Rewarded with top ratings(RE) twice. Worked at Ads team, contributed to Inventory Estimation and Realtime Processing Infra.- Help the team launch the next generation inventory estimation engine.- Save the offline pipeline for Inventory Estimation by 9k CPU days per day.- Reduce the memory footprint by 50% and latency by 30% for Inventory Estimation; Reduce the memory footprint by 85% for Ads conversion estimation system.- Develop the cache layer for Realtime Processing Infra.- Build Ads Signal A/B testing platform: work with 6 teams in Ads to support A/B testing for events.

Menlo Park, California

Database team. I was working on DocStore project in the MySQL team. This project introduced a new type called document to MySQL, which supports storing JSON in MySQL. Implemented partial updating and re-architecture the code of the project.

Beijing

Work on Web Data team. Focusing on ML and graph database.Build the models for different purposes: - Identify empty web pages. Improve the recall rate by 150% while keep the precision the same. - Identify the video cheating web pages with more than 95% recall and precision. - Identify the search result web pages.Design and implement a distributed knowledge database system. - Lead the design of the backend storage and frontend query language. - Implement the MVP features and launch to production.

Guangzhou, Guangdong, China

Master Of Computational Data Science

Bachelor'S Degree