Advancing the field of Offline Reinforcement Learning.Reinforcement learning (RL) presents a general framework for optimizing sequential decisions, central to many real world applications. There is significant interest in RL, yet the impact of RL research on the real world (e.g., recommendation systems, healthcare, robotic tasks) has been limited. A key challenge is that most RL algorithms assume the availability of active interactions with a live environment or simulator. Applying these RL algorithms to complex real world problems is prohibitive due to the difficulty of building a high-fidelity simulator as well as the cost and risk associated with collecting interactions in the live environment. To tackle this challenge, I have:* Established the first standardized Offline RL benchmarks (D4RL) that has been widely used.* Developed state-of-the-art Offline RL algorithms that are widely used in research and applications.

Conducting academic research on reinforcement learning and variational methods. Mentoring junior researchers in related areas. We have:* Greatly improved the robustness of the state-of-the-art off-policy continuous control algorithm (https://arxiv.org/abs/1812.05905).* Improved the sample efficiency of continuous control algorithms with a hybrid model-based/model-free algorithm (NeurIPS 2018 oral presentation <1%).* Derived a much more efficient drop-in replacement for the standard multi-sample gradient estimator in latent variable models (ICLR 2019).

High variance gradient estimates are a key challenge in many areas at Google, from interpretable models to practical reinforcement learning for robotics. I have led an effort to improve our understanding of variance reduction strategies and their applications. Moreover, I serve as a “go to” person on the Brain team and beyond for variance reduction strategies and discrete latent variable models. * Two papers accepted to NIPS 2017 (1 for an oral presentation, 1.2 % acceptance rate)* Mentored two junior researchers resulting in publications at ICLR 2018 and ICML 2018

Conducted machine learning research:* Best Paper award in the ICML 2017 Deep Structured Prediction workshop for our variational bounds for sequential models* Presented work on a novel regularizer for neural networks at BayLearn and ICLR 2017 Workshop

Developed statistical models for small-footprint keyword spotting on Amazon Echo, Dot, and related products. I contributed to all parts of the modeling stack from building tools to accelerate data collection to model development to on-device QA.* Released multiple models that improved the customer experience (as measured by internal metrics).* Mentored and managed two graduate student research interns.* Shared results with the research community through publications at SANE 2015, Interspeech 2016, SLT 2016, and NIPS 2016 workshop.

Developed novel statistical methods for risk prediction and association testing that significantly improve statistical power and computational efficiency over the state-of-the-art. Results published in Genetics, Nature Genetics, and AJHG.

Designed and implemented statistical models to infer protein-protein interactions and signaling networks from diverse experimental data. Used regularized regression, Gaussian processes, and other probabilistic models. Results published in Cell and Science Signaling.

Prototyped a locality sensitive hashing (LSH) scheme for fast item similarity searches. Demonstrated significant improvements in item recall quality over the existing system.

Designed and implemented a recommendation system using Vowpal Wabbit and Pig. Demonstrated significant performance improvements over the existing system.

Implemented a pipeline for evaluating models and tuning model parameters in R. Used the pipeline to develop an automatic gender recognition system based on telephone audio that was more than 90% accurate and robust to age and different call systems.