Making cars "see"

Co-lead of a ~40-ppl team in Google Research working on Image and Multimodal Understanding. Our team develops and owns Google-wide, product-oriented CV and multimodal signals from close- and open-vocabulary classification, object detection and segmentation, to vision and multimodal representations (embeddings). These signals are used in nearly all products that touch pixels, and power 100+ product functions and features. To name a few: - Google Lens (all non-OCR features including visual matching and recognition of extremely fine-grained) - Google Photos (search and other smart features) - Image Search (core ranking and search features) - YouTube (frame-level and thumbnail features for downstream ML) - Ads (policy classification and performance modeling) - Shopping (taxonomy classification) - Cloud (vision API and enterprise solutions) - Waymo (Content Search) ...I'm also one of the cross-team leads on large vision and multimodal models. Goal of the research is to advance the state-of-the-art of vision/multimodal and to explore novel capabilities such as search with multimodal inputs and multimodal conversations.

Worked on Google Search:- Developed DNN-based image embedding for Search-by-Image.- Developed DNN-based image annotation model for personal photo search.- Led a multi-team multi-quarter effort to migrate the image annotation+embedding model to 1st-gen TPU (first vision model migrated).- Worked on quality and infracture of content-based image search ranking and search-by-image.

Worked on: - Generative and discriminative models for person verification - Soft-biometrics (age/gender/...) - Metric learning - Retrieval & ranking

Worked on video classification.

Worked on multimedia retrieval.