I am a highly skilled Principal Platform Engineer / Architect with a passion for creating innovative, scalable, and efficient solutions in the artificial intelligence domain. My role involves architecting, developing, and maintaining platforms for our cutting-edge AI products.Key accomplishments during my tenure at Blattner Technologies & BOSS AI:🚀 Led the successful migration of the platform from legacy servers to a Kubernetes and fully cloud-native environment, utilizing helmfile for effective management of helm deployments across multiple clusters.⏱ Significantly reduced platform installation time from 2 weeks to 3 hours, enhancing efficiency and accelerating time-to-market for our AI solutions. I developed CI/CD pipelines to automate the deployment process, providing repeatability and change management through various environments.🔒 Implemented OAuth/OIDC across the platform and product, ensuring secure and seamless authentication and authorization for our users and machine-to-machine workloads alike. Migrated backend user management from a SIMP-based LDAP+CAS environment to Keycloak with native OAuth/OIDC.🏗️ Designed and improved Gitlab CI/CD pipelines for our product and its various components, resulting in streamlined development processes, automated testing with Apache Airflow, and faster releases.🤖 Spearheaded the adoption of Kubeflow as an AI/ML development and hosting platform, enabling our team to effectively build, train, and deploy machine learning models in a consistent and scalable manner with a focus on tighter security through OAuth and Istio service mesh.🖥️ Developed internal tools for cluster monitoring and management, empowering our team to swiftly troubleshoot and optimize performance.🗄️ Deployed and maintained a variety of data sources and utilities related to our product offerings, including PostgreSQL, OpenSearch (ElasticSearch), MSSQL, and S3, along with Apache Nifi for data ingest and ETL.

When I started at Mozilla, systems administration in IT consisted of a single small team of only 5 sysadmins. My team owned management of most of the company’s internal and external facing services. Over a 2.5 year period of rapid growth, I built this group up to 28 people and created 6 teams that could each focus on a functional area or set of services. These areas include core data center infrastructure, corporate IT, database administration, external web hosting, storage & virtualization, and developer services. These teams consist of a diverse workforce spanning multiple time zones and continents, with a majority working from home.During this period I made improvements in the infrastructure through building documentation and training processes on the people side, and redundancy in services on the technology side. I worked closely with business owners and developers outside of IT to meet their needs and deadlines in our hosting environment, which involved the installation and management of hundreds of servers across dozens of services. In the corporate IT environment we scaled our communication systems to meet the demand of growth from 300 employees to over 1,000 in just a couple of years. I am responsible for the budget and direction of our physical hosting facilities as well as developer workloads both on-prem and in the cloud.I like to help develop and deploy these solutions when time allows, yet my management style provides my team with the trust and autonomy they need to get their work done in the best way possible.

* Linux High Performance Computing (HPC) cluster sysadmin* Full-stack cluster installation and support* RedHat Enterprise Linux (RHEL) / SuSE Enterprise Linux (SLES) administration and deployment* Data center migrationIn this role I was a systems administrator for the Indiana University "BigRed" supercomputer, an IBM bladecenter cluster with 1,000 JS21 PowerPC blades managed by xCAT, and supported by multiple clustered filesystems (GPFS, Lustre, NFS). I tested and deployed new software, system updates, and responded to issues both in hardware and software. I assisted in the troubleshooting and resolution of job scheduler issues. When not working on the main clusters, I developed proof-of-concepts for new clustering and grid computing technologies through the FutureGrid project, and supported technical resources for arctic ice research through the PolarGrid project.

* 700 node system deployment across US, Singapore, Germany* Customer service* Rapid growth in infrastructure and peopleManaged a group of IT Engineers who ran the network, managed services, application development, and internal IT needs for a global ISP and hosted services provider. Through my short time there I managed the growth of the IT organization to over double its original size, and implemented processes, procedures, and documentation to help further growth. This was done while I was managing the successful deployment of a large-scale global infrastructure on a tight deadline for a customer.

* Data center management & remote-hands support* Mentoring / training of student sysadmins * Remote collaboration with FLOSS developers worldwideI managed a team of 7 Linux System Administrators at the OSU Open Source Lab (osuosl.org) soon after it was first created. We provided managed and unmanaged hosting for many open source projects, including but not limited to: Linux Kernel, Mozilla, Gentoo Linux, Debian Linux, OpenOffice.org, Apache Software Foundation, Drupal CMS, Freenode irc network, and more. I raised funds to sustain the lab’s free hosting through building relationships with key partners like Google and IBM.My daily activities ranged from administration of our internal and managed servers, shared services (DNS, MySQL db servers, SMTP relays) to managing the physical operations of the hosting center which housed all of our hosted projects.

* Linux and Irix administration and support* Taught Linux administration courses In this role I maintained multiple Linux servers as the primary administrator, including a high-capacity, high-availability file server. In addition to my sysadmin duties, I was available every day to provide support and consultation for Linux and Irix users across all 8 campuses of the Indiana University system. These requests ranged from needing to solve problems and fix broken systems to consulting on new projects, such as converting a lighting simulation lab in the theater department from Irix to Linux to help save on costs while improving performance.As I grew in the position and started working with the HPC team, I installed and managed a 2,000 node Condor cluster, then designed a web based job portal for the cluster in PHP. This cluster made use of ephemeral resources across all of the computer labs in the campus to harvest spare cycles in support of genomic research.As part of my consulting role, I also taught Unix systems administration certification courses through IU’s EdCert program and interfaced with some open source communities and helped to colocate services for them in IU’s data center.