Ads ML Infra

Led the Infrastructure Team (80+ Engineers) in CloudwalkWorked on data infra, deep learning infra, AI cloud service, and cluster-management platform for on-premise AI service.

Worked on scalability, data security and data ingestion for data infrastructure including HDFS, Presto, Spark, Flink, Druid, and etc

Worked on resource provisioning and management for AzureDBImplemented various techniques to improve the resource efficiency for AzureDB.

• Studied the novel problem of how a single server RDBMS can leverage available remote memory in the cluster accessed via RDMA to improve query performance.• Abstracted remote memory, accessed via RDMA, as a lightweight file API. • Implemented and integrated the abstraction into SQL Server.

• Investigated the problem of CPU sharing and isolation in a multi-tenant DaaS.• Designed a dynamic priority scheduling algorithm to isolate tenants’ resource requirements, while accommodating the notions of pricing, penalties and differentiated service quality• Implemented a prototype in Microsoft SQL Server and evaluated the workloads with variety of workloads.

Designed and implemented a new buffer page replacement algorithm for multi-tenant scenarios involving SLAs and overbooking.

Worked on the data exploration project and developed a greedy algorithm to mine size constrained set cover on real-world data set.