I founded and lead the centralized Evaluations org of research scientists and engineers that is responsible for evaluation strategy, development, tooling, and execution for all GenAI foundation models (e.g. Llama). Our work spans all modalities (text, image, audio, video, etc.), capabilities (safety, reasoning, intelligence, agentic behaviors, etc.), and evaluation types (automated, AI-assisted, human, etc.).

I led the company and applied research teams developing the technologies for customizing CV and NLP foundation models (LLMs) by focusing on the critical ingredient: the data. Snorkel AI has Fortune 500 customers in most verticals (including 7 of the top 10 U.S. banks) and was recognized in 2023 in lists such as the Fortune AI 50, Forbes AI 50, and LinkedIn 50 Top Startups.Leadership & Strategy-- Guided the company through stages of high growth, economic recession, and an AI boom-- Defined company-wide AI strategy, identifying opportunities and emerging industry trends, setting priorities, etc.-- Raised $135M from top-tier investors: Blackrock, Addition, Lightspeed, Greylock, Google Ventures, SV Angel, etc.-- Managed hybrid teams across four time zones consisting of full-time employees, contractors, and affiliated researchersResearch & Technology-- Defined the core concepts, terminology, and workflows for weak supervision used industry-wide and taught in ML courses-- Developed and launched Snorkel’s first generative AI offering, driving 30% of company ARR within 6 months of kickoff-- Led research initiatives in programmatic data development, LLM alignment, synthetic data, etc. for both NLP and CV-- Oversaw the large-scale distributed training of multi-billion parameter models on up to 10TB+ of data with 40+ GPUsTalent & Culture-- Designed interview loops, closed candidates, and ran 100s of interviews for tech, non-tech, and executive roles -- Fostered a diverse and inclusive environment with twice the national average percentage of women on AI research teams-- Organized and ran weekly team meetings, bi-weekly company all-hands, and quarterly workshops developing research skills-- Recognized on Forbes 2023 Best Startup Employers list based on employer reputation, employee satisfaction and growthhttps://snorkel.ai

Machine learning systems, large language models (LLMs), weak supervision, multi-task learning, training data curation, and learning from natural language with Chris Ré and Percy Liang in the Hazy Research Group, Stanford InfoLab, Stanford DAWN, Stanford NLP Group, StatsML Group, and Stanford Artificial Intelligence Lab (SAIL) during my PhD.More details on individual projects on my personal website: https://bradenhancock.com.*https://snorkel.orghttp://ai.stanford.edu/http://hazyresearch.stanford.edu/http://dawn.cs.stanford.edu/https://nlp.stanford.edu/http://statsml.stanford.edu/index.htmlhttp://infolab.stanford.edu/

Generative AI, improving chatbots with continual learning from interactions with users post-deployment. Mentored by Jason Weston and Antoine Bordes as a part of Facebook Artificial Intelligence Research (FAIR).https://research.fb.com/category/facebook-ai-research/

Reducing hallucinations in generative AI search results for semi-structured web content. Joint project with WebAnswers, Structured Data, and Google Brain groups.https://ai.google/research/teams/brain

Recommender systems for DoD applications and computation on encrypted data in the Computing and Analytics Group.https://www.ll.mit.edu/

Multi-objective optimization algorithms (evolutionary and gradient-based methods) in the Design Research Group.http://design.byu.edu/

Natural language processing (NLP), machine learning, social media mining in the Human Language Technology Center of Excellence (HLTCOE).https://hltcoe.jhu.edu/

Evolutionary optimization methods, airfoil design in the Turbine Engine Division.