EM in Persistent Disk (block storage), Google Cloud Platform

EM for the SCOPE Job Manager (aka job scheduler) team in Cosmos.We rebuilt SCOPE JM on Apache Tez which led to significant improvements in job performance, reliability, and cluster utilization. Supported the team through PoC, development, and customer migrations as we went from 0 to more than 500k daily jobs on the new runtime.

Scheduling improvements for scale up of Apache YARN Resource Manager And helping the team run one of the largest Apache YARN deployments in the world!

Added infrastructure support for Linux, Docker and OSS workloads.Contributed to opportunistic execution in Apache YARN.

Technical lead in Azure Analytics for NRT data processing (in minutes) and reliable data movementDesigned and built the NRT pipeline which powers Azure Resource Health (ARH). ARH uses an NRT application which processes nearly 300GB an hour of data from Azure resources and does stateful transformations to compute resource health. Led a team of 4 engineers on this effort.Designed lambda architecture for ARH to combine results of streaming analytics with that of batch analytics.Designed and built the Spark streaming based NRT analytics platfrom for Azure services (internal/first party). This solution integrated into Azure diagnostics pipeline and downloaded data from Azure storage and EventHub, transcoded from Microsoft Bond format into Spark data frame, and allowed publishing data into the datamarts for visualizations. Designed and built a highly reliable path for moving data at scale with zero data loss (billing for e.g.). A smart client and service were developed which sent data on a set of distributed channels (Azure Eventhubs or Azure storage) and in case of failure it fell back on a different channel. The solution was integrated with monitoring so we could detect failures at any time and alert the oncall engineer.Designed and built a scalable and reliable data collection service (pull model). The service today collects 100s of TB a day from systems that can’t push data to the analytics platform.Worked on batch analytics solution for VM uptime calcualtions. Prototyped a log clustering solution for mining anomalies and service patterns from logs in the batch pipeline.

Worked in the Exchange Enterprise Cloud monitoring team.Designed and developed a service to monitor network connectivity of from Exchange Enterprise Cloud from customer premise.Designed and prototyped feasibility of Virtual Machine usage for Exchange Auxiliary roles in the data center stack (it covered hands off allocation, deployment, recovery, and management for host and guest machines).

Working in the Exchange Enterprise Cloud monitoring team.Redesigned the monitoring service for stability and redundancy. The service is used to execute change management and remediation scripts against thousands of servers.Implemented features in the Silverlight portal used by the service engineering team for running scripts via the monitoring service.Worked on outside-in monitoring solution using Azure.

Worked on Amalga v3 in the Fabric team.Focus was on performance and scale. Identified major perf bottlenecks and drove changes for improvements in those areas (data access layer, WCF perf and scale, caching at different levels, etc).Test owner of the Amalga v3 application SDK. Wrote test applications, drove features into the SDK to make dev experience better.

Worked in the Windows Sustained Engineering security team and fixed issues in Winlogon, UAC, AppLocker, AxIS, and Kerberos. Shipped many QFEs, GDRs for Vista, Win 7 in SP2, SP1 respectively.Did a lot of debugging on all sorts of problems (race conditions, deadlocks, memory corruption, etc) with WinDBG.

Windows Sustained Engineering (security team)

Worked in the SIP team on the initial version of NT service for VoIP functionality (basic calling, conference, snooping) on PJSIP stack. Debugged NAT traversal issues, one way audio/no audio, etc.