Worked closely with executive management and senior engineering to modernize an aging and legacy data platform to provide an agile and scalable foundation to build from growing to 13m MAU and 5m DAU- Led team to plan, build, and execute the modernization of the data platform from a legacy self-hosted solution to Snowflake+Databricks. The migration reduced costs, increased performance of pipelines and analysis, and provided the key foundation for an scalable long-term platform to meet the growing needs of the business- Led a cross functional team to implement an experimentation and analytics platform to increase insights into key business metrics (subs, ltv, churn, pricing, etc..) and reduce the time to make key decisions- Developed an algorithm and pipeline to identify real users from ephemeral/spam/bots, providing the ability to gain more accurate insights into user segmentation and behaviorProject Details:Data Platform Modernization Project:* Reduced overall costs by 25%* Migrated multiple petabytes of data out of CDH+Hive in S3 to Snowflake* Increased performance of key pipelines by 50%* Cut ML development and deployment time by 50%* Built and mentored team to implement spark streaming with KafkaExperimentation & Analytics Project:* Increased ability to run experiments from ~5 per month to 20+* Reduced time to StatSig from multiple months to 2-3 weeks* Increased visibility into the business for quicker decision making leading to $10m+ in extra revenue* Built taxonomy to track and curate events/attributes allowing for ~30% reduction in eventsReal User Algorithm Project:* Developed using spark+graphframes and the connected components graph algorithm to combine users and devices to create user ←→ device relationships* Prototyped moving from daily batch with spark+graphframes to a graph db like neo4j to leverage the capabilities of a graph db and expand the use cases into behavior analysis

Rejoined Flickr to drive a modernization and stability effort which grew into include building and managing a new data team- Led team that migrated all of flickr's infrastructure out of on prem yahoo datacenters into AWS which included over a dozen different backend and frontend microservices and ~1k physical servers into AWS- Developed a simple yet effective similarity search service using Flickr search + ANN (approx nearest neighbor) to improve search ranking. This provided the foundation for building a youtube like engagement experience where similar photos would also be shown on photo pages from search- Developed modern approach to generating Flickr's 20+ billion photo sitemaps using spark + kafka reducing the cost by 75% and improving the generation time from 18hrs to 1hr- Rebuilt search feeding pipeline from a legacy java+redis app to modernized spring boot app + kafka capable of handling 250k search insert/updates per second. Improved user experience reducing availability in search from ~30m to ~1m and also reducing cluster rebuild time from one month to just three days.- Rebuilt Flickr authentication as part of the migration from Yahoo Membership to AWS Cognito reducing login related issues by ~25%- Managed a multi-disciplinary team of engineers on backend and data. Responsible for career development, coaching, project management, and architectural guidance. Defined new roles and structure for the data/ML team to build out new features in personalization, search, analytics, and behavior analysis.

Part of the Flickr team which was spun off into a Yahoo platform team named Tripod. The directive for this new team was to provide an internal media (photo+video) platform that cherrypicked three key verticals from Flickr - Media Management (upload, serving and storage), Computer Vision/ML and Search- Led the backend/ML team that developed the computer vision/ML components of the internal Yahoo media platform “Tripod”. Refactored backend and ML services from Flickr to support labeling, face detection, ocr, similarity and nsfw for Yahoo Mail, News, Sports, Messenger and Yahoo.com

- Drove effort to build out first ever computer vision pipeline at yahoo. The initial pipeline had 100 classifiers which eventually grew to 1k and was based on using storm for realtime processing- Collaborated and built out internal yahoo multi tenant media search platform based on sharing publicly tagged images from Flickr within Yahoo- Developed modern search implementation to use updated search language from https://vespa.ai which allowed search to be less obscure and easier to understand across the BE team

* Technical lead for a team developing a private and public cloud based architecture for storage and compute resources. * Implemented Nagios to centralize the monitoring of core systems and infrastructure which had been monitored via custom scripts and cron. * Implemented custom package build system for Red Hat and Solaris systems. This allowed for faster and easier deployments by including these packages in in the local kickstart/jumpstart environments along with our custom deployment platform.* Technical Architect for homegrown software management and deployment platform. This platform allows users from development, QA and operations to build, test and deploy code and applications to their respective environments. This cut deployment time and testing from days and months to minutes and hours. * Technical Architect for the migration from old VMware server and physical server infrastructure to VMware ESXi. This allows for better use of infrastructure resources while also providing better redundancy in the event of any hardware failures. This dramatically reduces the support and environmental costs from the current large installation footprint.* Built a VMware ESXi addon module for the homegrown software management and deployment platform that uses the VMware Java SDK to pull data from ESXi and vCenter. The integration includes being able to power on, power off, reset and provision new virtual machines within the management platform. This gives users quicker access to the virtual machine controls as well as removes the necessity of the user to console to the specific VMware ESXi server to perform these tasks.* Spearheaded an initiative to architect and move many production servers into Amazon’s virtual private cloud. Using Amazon’s API’s I was able to integrate the automated provisioning and fully utilize the elasticity of the cloud for the business.

* Led team of six Network and Systems Engineers that experienced year over year decrease in unplanned production downtime and an increase in availability that aligned with the contractual six sigma SLA’s from customers for uptime, availability, and performance. * Successfully upgraded and deployed infrastructure for US operations including the automated fair collection projects for the Bay Area TransLink®, Seattle ORCA Card, and the regional call centers in Washington DC and Boston. * Lead Engineer for the successful design, build and implementation of updated storage architecture leveraging emerging SAN storage technologies (HDS 9570 SAN).* Engineered custom Nagios installation that integrated with HP Openview allowing customer service representatives to easily navigate and troubleshoot first line support issues* Heavily involved with automation projects utilizing shell scripts, PHP, and Perl (software verification, embedded device verifications, software deployment, device monitoring, etc…) * Successfully designed and implemented redundant mail gateways used by US based projects.* Participated as part of a team that was key in the collaboration and development of the first regional disaster recovery and business continuity plan(s) in the entire organization. The plan covered all aspects for the US operations and projects including technical, financial and people.* Responsible for maintenance and support of all configurations and modifications to the central Translink system & software (Change Management). * Developed and implemented a jumpstart server strategy at regional collocation facilities and local offices. This included developing custom profiles, drivers and post installation scripts for both sparc and x86 platforms. The implementation was key to the constant deployment of the seven test beds that were used to meet the aggressive dates for several US projects.