Brian Witt Email & Phone Number

Mountain View, California, US

San Francisco, CA, US

• Member of the Reliability team
• Led the effort to prepare all services to handle 5-10x normal traffic, identifying key metrics andstakeholders, developing mode reports, and working with stakeholders to prioritize work.
• Implemented automatic SLO error budget burn rate alarming, enabling product engineeringteams to remove alarms and depend on these instead, lowering mean time to detect outages.
• Designed and implemented a Python and PostgreSQL (RDS)-based ranking system to showreliability problems with services, enabling owners to make better reliability choices.
• Educated and helped product teams set good SLOs for their services, enabling them to makebetter choices regarding reliability vs feature work during planning.
• Analyzed severe incidents to ensure proper investigation had taken place, find common threads,create plans to address common issues and ensure issues are fixed.

San Francisco, CA, US

• Member of the Developer Experience and Source Control team
• Designed and delivered a pre-provisioned development environment to internal engineers,saving hundreds of hours of developer time.
• Designed the strategy to implement GitHub Enterprise behind Duo authentication but still allowautomation to function.
• Led efforts to improve the reliability of the dev environment, solving common problemsengineers were encountering, saving many hours of development time.
• Improved onboarding process and education on current development infrastructure.
• Scaled automatic container parent update system to efficiently push updates in a timely manneracross the container image tree, avoiding potentially costly security issues and outages.

Sunnyvale, CA, US

• Promoted to lead tech ops strategy, design, and priorities for BrightRoll Video Ads Production EngineeringVerizon acquired Yahoo in June 2017, combining AOL and Yahoo into a new entity called Oath.Tech Yahoo.
• Enabled scale-up and growth of the ad-serving business, supporting ~300k QPS, driving steadilyincreasing revenues.
• Managed 1500+ AWS VMs and bare-metal servers with Puppet, MCollective, and Foremanimproving team efficiencies and managing workloads more efficiently, reducing risk, andrecovering more quickly from server outages.
• Led project to move credentials into encrypted storage to increase security, decreasing riskwhile meeting business requirements.
• Owned Puppet code repository ensuring quality and maintainability.

Sunnyvale, CA, US

Sunnyvale, CA, US

• Yahoo acquired BrightRoll in 2015. Continued role leading tech ops for BrightRoll Video Ads Production Engineering.Tech Yahoo, Service Eng, Sr
• Revamped internal puppet support of MySQL databases, allowing for quicker provisioning andlower downtime.
• Migrated 18 MySQL database servers from third-party hosts into DCs, saving ~$20k / mo.
• Developed policies for use of VPCs in AWS.
• Implemented centralized storage of config data for the puppet system, improving manageability.
• Implemented hardware monitoring and replacement procedures for DC servers, reducing MTTRfor hardware-related issues by 36%.

Sunnyvale, CA, US

• Yahoo acquired BrightRoll in 2015. Continued role leading tech ops for BrightRoll Video Ads ProductionEngineering.
• Re-wrote MySQL database puppet config, allowing for quicker provisioning and less downtime.
• Led project to migrate 18 MySQL database servers into DCs, saving ~$20k / mo.
• Developed policies for use of VPCs in AWS.
• Implemented centralized storage of config data for the puppet system, improving manageability.
• Implemented hardware monitoring and replacement procedures for DC servers, reducing MTTRfor hardware-related issues by 36%.

Sunnyvale, CA, US

San Francisco, CA, US

• Migrated servers from AWS into datacenters in the US, Europe, and Hong Kong to support the capacity and latency needs of the business.
• Managed 1500+ AWS VMs and bare-metal servers with Puppet, MCollective, and Foreman improving team efficiencies and managing workloads more efficiently, reducing risk, and recovering more quickly from server outages.
• Led project to move credentials into encrypted storage to increase security, decreasing risk while meeting business requirements.
• Owned Puppet code repository ensuring quality and maintainability.

San Francisco, CA, US

• Automated archiving and purging of MySQL data to s3, preventing disk space issues.
• Implemented per-team alerting in Icinga, ensuring owners receive alerts directly.
• Automated management of AWS security group ACLs.
• Implemented security measures such as key-only ssh, locking down ports, and limitingpuppet access.

Founder at a startup developing rewards systems using smart cards and back office systems.