Apache Airflow is an open-source workflow automation and scheduling system that can be used to author and manage data pipelines. As one of its earliest maintainers, I helped bring the Airflow project into Apache and to grow its community, processes, and codebase.

I provide advisory services to startups, conferences, and incubators. These include:• Celect (acquired by Nike)• Data Council• DataZoom• Etsy• Niara (acquired by Aruba)• Prefect• Skills Matter• SparkLabs Foundry

QCon is a global family of practitioner-driven conferences with annual events in SF, NYC, London, Munich, Sao Paolo, Beijing, Guangzhou, Shanghai, & AI. • As a co-chair for QCon, I help curate tracks & talks for the conference, invite speakers & track hosts to participate, and promote programs and practices that spread social change for good in the software industry • As a track host, I design a track and invite speakers, sometimes speaking myself. I typically host tracks on data engineering, data science, and fault-tolerance.

Datazoom is a Real-time Video Data company. The Datazoom data platform collects and transits video telemetry data with high-fidelity and low-latency to a range of 3rd party data sinks. In my role, I contributed in the following ways:• Designed the "-ilities" into all of our systems• Built and managed a fun, healthy, and impactful engineering organization at Datazoom• Defined and oversaw the implementation of best-practices in all areas of engineering• Implemented various parts of our software infrastructure

As PayPal's first Chief Data Engineer, I led critical, high-value projects, helped grow the technical competence of the organization, and helped to promote PayPal’s external-facing technical brand. • Managed a team of data scientists developing recommendation algorithms as part of PayPal’s acquisition of Jetlore, a shopping recommender technology company• Led a team to design & build PayPal’s Streams-as-a-Service (known internally as the Core Data Highway or CDH). CDH offers users a self-service means to launch any data stream on-demand from any source (e.g. RDBMS, apps) to any destination (e.g. streaming consumers, persistent stores) with high-fidelity, low latency, & high-availability! CDH is currently used to stream up to 70K Oracle tables to a variety of target types.• Led a team to build PayPal’s Database-as-a-Service (DbaaS) platform in GCP. DbaaS offers a self-service portal that allows users to launch and manage DB clusters (e.g. CouchDB, MySQL, Aerospike)• Helped build a high-quality, high-performance software development practice focused on building software data infrastructure at scale (e.g. streaming systems, data stores, big data processing) in PayPal’s Core Data Platform org• Helped define a standard engineering hiring process for IC candidates up to Director-level ICs• Participated in the IC promotion panel for senior candidates up to and including Senior Director-level ICs• Helped grow PayPal’s technical brand by promoting a combination of conference speaking, open-sourcing, and blog writing across the company• Earned 3 patents in the areas of stream processing and graph databases• Helped open-source multiple projects at PayPal, including Yurita (Anomaly Detection), Gimel (Big Data Processing), Name Node Analytics (Hadoop Name Node Analytics), etc...

Agari tackles the problem of enterprise spear-phish detection and prevention. Agari was acquired by HelpSystems in May 2021. I joined Agari to help design and build nearline fraud prevention systems as a service.• Designed and built reliable predictive data pipelines in the AWS cloud for bath and near-real time control systems and analytics • Championed tech branding efforts at Agari (e.g. blogs, podcasts, conference speaking, open-source contributions)

I helped develop LinkedIn's novel Search-as-a-Service (Galene) system & search auto-complete (a.k.a. search type-ahead) backend. Typeahead is one of the 3 most visible ways that users interact with LinkedIn. First, a user searches for another user using typeahead, then views his/her profile, and finally connects with him or her. Search type-ahead is federated across both graph (e.g. members) and non-graph (e.g. companies, groups, schools, skills) search indices.• Co-led a 20+-person effort to create LinkedIn's novel Search-as-a-Service (Galene)• Led a small team to build search type-ahead (a.k.a. auto-complete) on Galene• Led the search Indexing team, responsible for building offline (Hadoop) and near-real time indexes

I joined the Distributed Data Systems (DDS) organization at LinkedIn shortly after LinkedIn's IPO. This organization created and supported online, nearline, and offline data systems at LinkedIn. These included Kafka, Databus, Espresso, Voldemort, etc... DDS also owned all data landing in Hadoop for ETL into data warehouses, for indexing in specialty systems (graph, search, recommenders), or for use in various data science & analytic flows. There were 80 people in the org when I joined.I was the technical lead for the Analytic Platform (AP) team, which was responsible for managing data ingest into Hadoop and the Data Warehouse and for making this data available for analyses via the Hadoop ecosystem or via traditional reporting and data warehousing tools. • Led the development of LISTT (LinkedIn Segmentation & Targeting Tool). LISTT is a self-service tool that marketing operations uses to target LinkedIn members for online marketing needs. It does this by leveraging both Hive and Pig to materialize a large table in Hadoop. This table is then converted into multiple formats: Teradata load-ready format & Lucene indexes for a custom search application • Presented LISTT at Hadoop Summit 2013

I was the first engineer at Netflix to work solely on the public cloud as establishing a hybrid data fabric was the first major hurdle to cloud adoption. My goal was to get Netflix's data into the cloud and to keep it synchronized between Netflix's data centers and the AWS public cloud.I led all aspects of data infrastructure during Netflix's entry into the cloud as a leading member of Netflix’s Cloud Platform team.• Part of a 4-person sub-team that designed and built the world’s first cloud-based video streaming service • Brought NoSQL to Netflix• Helped build Netflix’s Cloud-based Data Infrastructure (e.g. Cassandra, SimpleDB, S3, etc...)• Filed the first 2 patents involving Netflix's new Cloud-based Data Architecture• Authored a white paper titled “Netflix’s Transition to High-Availability Storage Systems”• Evangelized NoSQL, Data Replication, & Cloud Best Practices internally and externally• Served as a Netflix Crisis Manager – lead the resolution of critical company-wide production issues• Championed Netflix's early efforts in tech branding, setting the stage for its current program

I joined Netflix when there were ~100 people at the company. As a founding member of Netflix's Software Infrastructure team, I helped define, design, and implement core libraries, services, and infrastructure – all Netflix systems run on our infrastructure. I also solved critical performance problems, resulting in cost reduction and service improvement.• Led the identification and resolution of various Denial-of-Service exploits. Evangelized DoS prevention. • Found and eliminated a critical performance bug in the streaming PC player – this fix reduced DB traffic by 50% to 2 key tables, reducing our need to vertically scale the database • Increased farm-wide memory headroom by 10% by eliminating the use of a Java Finalizer• Created Netflix’s Session Manager to deliver consistently fast user response times under high traffic• Created Netflix’s web request processing framework to improve developer productivity and code robustness• Created a deadlock recovery system to detect production deadlocks and take preemptive action before end users could be affected• Invented 2 internal performance optimization tools (i.e. Tracer Central & Tracer Regression Central) – Netflix Engineering relied on these tools to understand traffic growth and code/site performance

I started advising the founders of Etsy when they had first started. A year later, Rob Kalin asked me to join Etsy as employee #12 and to run Engineering as Etsy's first VP of Engineering. • Managed a distributed engineering team• Doubled Engineering through recruiting / hiring• Designed and wrote a real-time application logging and analytic application• Helped design the database data models and layout• Helped offload database load over to search via Change Data Capture

Hot off my successes at the eBay Research Labs, I was invited to join eBay’s Search Engine team to work on a variety of cool search systems• With help from a co-worker, I built an in-memory database for buyer behavior, central to eBay's new search ranking system (i.e. a second-pass ranker called Best Match). • Implemented a search service to find (fuzzy) near matches for sellers by user id, first name, last name, or full name

After I submitted 7+ innovation ideas to the eBay Research Labs, eBay started an Innovation Rotation program which allowed engineers to rotate into the Research Labs to prototype their ideas. I was given 9 months over the standard 6 months to focus on 3 different projects.As a member of the Ebay Research Labs, I had the opportunity to work on various early-stage prototypes:• Partnered with a colleague to build a WYSIWYG eBay Store builder leveraging the latest browser-side technologies. The goal was to build an engaging eBay Store flow as a responsive single page implementation.• Designed & implemented a novel P2P version of eBay over Skype. This new version introduced various distributed system state-machine challenges, which I solved with a novel distributed ledger implementation.• Attempted to build a single-cpu cycle fast filter that could be used for recommenders & search ranking

My career at eBay started as a founding member of the Stores & Merchandising team. The dual charter of the team was to build recommender systems and pave the way for eBay Stores (i.e. online seller storefronts hosted on eBay). My work at eBay involved touching every part of eBay.During my tenure with this team, I led projects touching functionality across all areas of the site.• Led projects that touched Search, MyEBay, Selling, Buying, Sign-on, API, etc...• Proposed & led a project to re-architect the eBay subscriptions framework• Introduced AJAX at eBay! It was a budding new technology at the time.

Siebel Systems created the CRM market in the late 90s and dominated the market in the early 2000s. Software was sold to customers who were then expected to install it on their servers -- this is well before the advent of the SAAS CRM business model.Siebel Server code (written in C++) was advertised to run on commodity hardware with low latency at a specific concurrent user load. As Siebel applications added more features, performance & scalability degraded. It was my team's job to win back CPU and memory headroom through algorithmic & data structure modifications to meet performance and scalability targets. This involved all of the following:• Ran repeatable & precise scalability benchmarks to quantify scalability degradation• Executed memory and CPU profiling followed by profile analysis• Made code changes to regain scalability headroom