GenAI, Gemini, multi-modal generation and understanding

Generative video model==== Previous ====Fundamental research: segmentation, depth estimation, foundation model, video model, multi-task learning, scene understanding, self-supervised learning. Applied research: on-device ML to enable visually impaired people run independently.

Computer vision research for object recognition, image understanding and image segmentation.

Building a framework that generates FPGA and ASIC accelerators from a high level language (Halide) for deep learning applications.Developed an analytical model to explore the design space of Deep Nerual Network (DNN) accelerators, including the choices of loop blocking, dataflow and memory system.Performed design verification for a floating point chip, and build the test board for the tapped-out chip.

Introduction to VLSI System (EE271)

Extended, improved and trained an edge-adaptive filtering framework (BLADE) to perform high dynamic range imaging (HDR).Optimized the framework to achieve similar image quality as the state-of-the-art CNN based models, but use order-of-magnitude less operations and parameters.

Compressed various DNNs using quantization, pruning, and weight sharing.Proposed a hybrid quantization method using non-uniform precisions, reduced the model size to less than 1/6, with the same accuracy.

Built an infrastructure to analysis power, performance and area, optimized the synthesis effort of each hardware subunit.

Accomplished simulating crystal oscillator circuit and voltage regulator circuit using spectra