Designed new query platform that reduced costs by ~30% by replacing always-on, in-memory architecture with on-demand, horizontally scalable one. Led team of three to five engineers to validate cost projections, build the design, add test tooling, and roll it to production with minimal customer impact.Set direction for team of three senior engineers and delivered on multiple company objectives, including process and technical changes to reduce support and oncall, separate deployment of services, updating query platform with new security features and adding complex data model support.

Designed system for streaming data ingest of many 10s of millions of events daily using Terraform, API Gateway, Kinesis and AWS Lambda. Led team of three and built significant portion of design.Updated custom Spark manager and Apache Spark to return enhanced diagnostics around Spark app failures and make scheduling decisions using this data to increase reliability and reduce oncall alerts.Modified Apache Spark to enhance observability for Spark apps, including building a Datadog reporter, automatic heap dumps to S3 on OOMs, and adding request ids and other diagnostics to logs.Debugged and fixed network saturation issues and intermittent file permission problems that caused Spark app failures, increasing reliability, reducing app runtime, and generating fewer oncall alerts.Wrote custom join operator to Apache Spark that allowed specific queries to execute on large data sets without shuffles.Co-created metrics-based framework and tooling to add observability to the ingest and query platform. Implemented it using Prometheus, AlertManager, and custom system to alert on missing events. Built Scala libraries to track tasks in services.Rebuilt data-ingestion platform, moving from a Hive-based setup to a custom-metastore, increasing reliability and performance, and dropping ingests from days to hours.

Wrote desktop client P2P networking stack using multi-threaded components interfacing with C++/Java libraries. Implemented relay server and client to transfer data through firewalls. Designed signaling and protocol, and built a non-blocking server and client using Java NIO. Replaced third-party relay and reduced sync failures by orders of magnitude. Co-designed system to propagate ACLs to desktop clients. Developed a Netty-based pub-sub server and message queue with mutual authentication and custom client revocation to deliver updates. Dove into Netty internals to assess design implications. Migrated AeroFS VCS from SVN to git. Automated Java (ant/gradle) and C++ (qmake) builds. Reworked bash release scripts for maintainability and introduced CI into the build and release process.

Primary developer on two-person team that built and maintained a C++ FIX server for multi-million-dollar institutional trades.Designed and implemented a new version of the server using in-house C++ server framework that increased throughput and reduced trade latency by minimizing shared access and resource contention.Wrote DTrace providers and scripts to identify bottlenecks. Designed and implemented integration-test framework for legacy FIX server, making it possible to validate server correctness.

Designed and implemented a proof-of-concept context-aware web-services infrastructure.Created architecture that balanced implementation complexity in existing middleware vs. ease of use.Extended open-source ActiveBPEL WS-BPEL engine with context extensions.Wrote WSRF/WSN components in Apache Muse and Java EE 5 web-services in Glassfish.Paper describing design and implementation accepted to CASCON 2008:http://dl.acm.org/citation.cfm?id=1463818

Developed a mesh-networking routing implementation (AODV) in Java for Sun SPOT small devices. Co-designed and implemented a proof-of-concept GUI tool for visualizing packet flow in mesh networks.

Designed and built a server-side Java SMPP PSTN-to-WiFi call-handoff adapter for VON 2005 demo. Used customer feedback to productize implementation for next software release.

Debugged and fixed issues in embedded DRM library for Qualcomm chipsets using Lauterbach ICE. Developed light-weight parser in C for OMA DRM 1.0 headers. Wrote automated system and adversarial tests for implementation.