Director Of Infrastructure Engineering
Fort Collins, Colorado, United States
As the Director of Engineering and Head of Enterprise systems, I am responsible for all operational aspects of the Clearbanc production and pre-production application and services, as well as the software delivery and quality assurance processes. When I joined Clearbanc, they were experiencing significant production failures on an almost daily basis. The teams had been without effective leadership for some time and morale was at an all time low. Within 8 weeks we had eliminated all of the previous recurring production failures, added significant and useful monitoring, and as a result had the smoothest Black Friday weekend in the company’s history. I also stabilized the team and focused on creating an environment where my engineers could do their best work.During that same time, and on an ongoing basis thereafter, I began a process of organizational debugging, searching for the issues that caused the failures the company was experiencing. Much of this could be attributed to lack of DevOps ownership, mistrust between teams and unnecessary gatekeeping / CYA behavior. I set about to dismantle that by getting teams more directly involved with each other and shifting the conversations from failure and blaming to constraints and solutions, with a focus on developing the teams ability to collaborate to create reasonable goals and expectations of their counterparts.Some highlights from this role: * Implemented incident response and post-mortem process for ongoing improvement and failure prevention. * Implemented operational insights program for effective alerting, monitoring and technical insight to our platfoorm. * Improved software delivery processes, going from multiple delivery failures weekly to multiple smooth software releases daily. * Improved security and resiliency by evaluating the cloud infrastructure and implementing a redeployment strategy that corrected multiple security vulnerabilities and network communication problems.