Member of the Reliability team● Led the effort to prepare all services to handle 5-10x normal traffic, identifying key metrics andstakeholders, developing mode reports, and working with stakeholders to prioritize work.● Implemented automatic SLO error budget burn rate alarming, enabling product engineeringteams to remove alarms and depend on these instead, lowering mean time to detect outages.● Designed and implemented a Python and PostgreSQL (RDS)-based ranking system to showreliability problems with services, enabling owners to make better reliability choices.● Educated and helped product teams set good SLOs for their services, enabling them to makebetter choices regarding reliability vs feature work during planning.● Analyzed severe incidents to ensure proper investigation had taken place, find common threads,create plans to address common issues and ensure issues are fixed.● Tuned common default envoy request settings for services, ensuring the defaults make sense for90% of cases.● Debugged container and service errors on Kubernetes using logs in Kibana, tracing via LightStep,and metrics in Grafana. Pulled out learnings to apply across all services.● Planned and implemented service migrations to Kubernetes, reducing operational costs andimproving capacity.

Member of the Developer Experience and Source Control team● Designed and delivered a pre-provisioned development environment to internal engineers,saving hundreds of hours of developer time.● Designed the strategy to implement GitHub Enterprise behind Duo authentication but still allowautomation to function.● Led efforts to improve the reliability of the dev environment, solving common problemsengineers were encountering, saving many hours of development time.● Improved onboarding process and education on current development infrastructure.● Scaled automatic container parent update system to efficiently push updates in a timely manneracross the container image tree, avoiding potentially costly security issues and outages.● Optimized Dockerfile builds for service containers and rolled out linters, documentation, andPR-based checks to ensure best practices are implemented.

Promoted to lead tech ops strategy, design, and priorities for BrightRoll Video Ads Production EngineeringVerizon acquired Yahoo in June 2017, combining AOL and Yahoo into a new entity called Oath.Tech Yahoo, Production Eng, Principal● Enabled scale-up and growth of the ad-serving business, supporting ~300k QPS, driving steadilyincreasing revenues.● Managed 1500+ AWS VMs and bare-metal servers with Puppet, MCollective, and Foremanimproving team efficiencies and managing workloads more efficiently, reducing risk, andrecovering more quickly from server outages.● Led project to move credentials into encrypted storage to increase security, decreasing riskwhile meeting business requirements.● Owned Puppet code repository ensuring quality and maintainability.

Yahoo acquired BrightRoll in 2015. Continued role leading tech ops for BrightRoll Video Ads Production Engineering.Tech Yahoo, Service Eng, Sr● Revamped internal puppet support of MySQL databases, allowing for quicker provisioning andlower downtime.● Migrated 18 MySQL database servers from third-party hosts into DCs, saving ~$20k / mo.● Developed policies for use of VPCs in AWS.● Implemented centralized storage of config data for the puppet system, improving manageability.● Implemented hardware monitoring and replacement procedures for DC servers, reducing MTTRfor hardware-related issues by 36%.● Developed alert triage policies and procedures for the operations team to ensure consistent,timely support of operational systems.● Mentored engineers, helping them grow in their professional careers.

Yahoo acquired BrightRoll in 2015. Continued role leading tech ops for BrightRoll Video Ads ProductionEngineering.● Re-wrote MySQL database puppet config, allowing for quicker provisioning and less downtime.● Led project to migrate 18 MySQL database servers into DCs, saving ~$20k / mo.● Developed policies for use of VPCs in AWS.● Implemented centralized storage of config data for the puppet system, improving manageability.● Implemented hardware monitoring and replacement procedures for DC servers, reducing MTTRfor hardware-related issues by 36%.● Developed alert triage policies and procedures for the operations team to ensure consistent,timely support of operational systems.● Mentored engineers, helping them grow in their professional careers.

● Migrated servers from AWS into datacenters in the US, Europe, and Hong Kong to support the capacity and latency needs of the business.● Managed 1500+ AWS VMs and bare-metal servers with Puppet, MCollective, and Foreman improving team efficiencies and managing workloads more efficiently, reducing risk, and recovering more quickly from server outages. ● Led project to move credentials into encrypted storage to increase security, decreasing risk while meeting business requirements.● Owned Puppet code repository ensuring quality and maintainability.

● Automated archiving and purging of MySQL data to s3, preventing disk space issues.● Implemented per-team alerting in Icinga, ensuring owners receive alerts directly.● Automated management of AWS security group ACLs.● Implemented security measures such as key-only ssh, locking down ports, and limitingpuppet access.

Founder at a startup developing rewards systems using smart cards and back office systems.