Autonomous Driving, Deep Learning, LLM，Multimodal（Visual Language）Model，Embodied AI.Modulator and speaker on "Embodied AI" workshop, GOSIM Beijing, Oct. 17-18, 2024. The speech topic is "What data imperative for action learning in embodied AI?"Speaker and Panelist of Innovation Forum "The Age of Industrial AI Agents: Opportunities & Challenges"(https://sites.google.com/view/mipr2024/innovation-forums/innovation-forums-2), MIPR'24 (https://sites.google.com/view/mipr2024), San Jose, Aug. 7-9, 2024.“Levels of AI Agents: from Rules to Large Language Models”, arXiv: 2405.06643, May, 2024.Invited keynote talk on "Applications of large foundation models for autonomous driving", at the first workshop on Large Language and Vision Models for Autonomous Driving ( LLVM-AD), in conjunction with IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Jan. 8th, 2024"Applications of Large Scale Foundation Models for Autonomous Driving (Survey)", arxiv:2311.12144, Nov. 2023Talk on “Introduction of BEV Network’s Extension, 3-D Occupancy Network for autonomous driving”, Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Sept. 16, 2023 Talk on “Introduction of BEV Network, end-to-end perception for autonomous driving”, Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Sept. 9, 2023 Talk on “Discussion on Key Problems in Building a Data Closed-loop of Autonomous Driving”, at Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Aug. 19, 2023 "An Overview about Emerging Technologies of Autonomous Driving", arXiv2306.13302, June, 2023“Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies”, arXiv: 2006.06091, June, 2020.

Autonomous Driving‘s R&D: BEV e2e framework and data closed loop platform development.Extend the company R&D domain from simulation framework to data closed loop and automatic data annotation for autonomous driving platform.The first athour of the book "Autonomous Driving System Development" in Chinese, Tsinghua University Press, May 2024.

Worked at Zero-One Tech, a SAIC subsidiary. Algorithms research and development in visual perception (2-D detection, range estimation, self calibration, tracking), mapping & localization (semantic map element matchiing, map matching), prediction (NN based) and planning (data driven) & control, BEV end-to-end framework for perception and prediction in a spatial- temporal fusion mode for neural planning development, data closed loop platform with servers’ models distilled to vehicles’ models as well as data cleaning, selection, annotation and training etc.

Worked at Zhito Tech，a joint venture company of Plus and FAW.Autonomous Driving of Commercial Trucks (visual perception, LiDAR perception, self caliibration, tracking, range and speed estimation, BEV perception network motivated by Tesla AI day，sensor fusion, mapping & localization, semantic map element matching, interaction aware trajectory prediction, data drivenn decision making, motion planning & control, simulation & safety critical scenario-based testing, system engineering & safety degradation design，software middleware platform and closed loop data cleaning/selection /annotation/training optimization/deployment engine etc).

time period：3/2019-3/2022.Shanghai Institute for Advanced Communication and Data Science.

Autonomous Driving Research （perception, mapping and localization, behavior modeling/prediction, planning and control, simulation).Nominated as 2020 Distinguished Industrial Leader by APSIPA (http://www.apsipa.org/, Asia-Pacific Signal and Information Association).

Singulato Research & Innovation Center @ Silicon Valley, USA.Moderator of IEEE MIPR'19 (Mar 28-30, San Jose, CA) Innovation Forum "Towards Autonomous Driving": http://www.ieee-mipr.org/history/2019/data/towards_autonomous_driving.htmlWorking for L2 - L3 - L4 autonomous driving (perception, simulation, and HD map).

Perception team: LiDAR and camera-based visual odometry & SLAM, unsupervised multi-sensor calibration and registration, early data fusion of stereo/mono camera and LiDAR by deep learning , online camera calibration for inverse perspective mapping, vanishing point detection, road lane detection & tracking, vehicle detection & tracking and vehicle orientation/distance estimation etc.Issued 3 US patents for deep learning-based data fusion of camera and LiDAR.

Graphics Media Architect at VPG (Visual Parallel Computing Group): 6/15-6/16Panorama generation, spherical video rendering, image search, object detection, compiler optimization，face recognition (supervising intern from Prof. HT Kung's group, Harvard U) and Deep Learning ( data/model parallelism, scene categorization).Technical evaluation of "Replay Technologies" (in March 2016 Intel purchased), VR companies as Immersive Media, 3D4U/Voke (purchased by Intel in Dec. 2016), NextVR, JauntVR etc.Lead the project of accelerating CDVS with OpenCL programming, realizing 5.6x speedup on BDW Laptop vs CPU only.Computer Vision Architect at CCG (Client Computing Group): 6/14-5/15Stereo matching for FG/BG segmentation, visual SLAM, AR (visual + IMU fusion), image-based relocalization and Deep Learning (Denoising and SR).Collaborated with UK startup "Seene", for scene reconstruction with mobile (iphone). Note: "Seene" was purchased by Snap (Chat) in June 2016.Joint work of real-time camera tracking for Intel RealSense 3D sensor-based developer kit in Google Project Tango.Implementation of pseudo real time stereo-based depth estimation on Intel Atom mobile products.

Human visual perception and machine learning-based video processing to improve visual quality and compression performance: 1. Video denoising with nonlocal self-similarity; 2. Visual masking and machine learning (SVM)-based visual quality classification for bitrate reduction in video compression; 3. Object contour extraction by ensemble learning (random forest) with clustered shape features;4. Image decomposition-based detail and contrast enhancement.More details are given in my website.

R&D in the Algorithm Team of Digital Media Solutions Lab: 1) Video denoising (blocking/ringing artifacts), detail enhancement & upscaling with non local self similarity and sparsity properties; 2) Image matting & compositing by Bayesian/robust methods, interactive object cutout with grabcut, inpainting with shift map (graph-cut), local GMM classifiers-based contour tracking for rotoscoping with prediction by smoothed optic flow (estimated by a Total Variation-based method);3) Visual BoW-based indexing & search with kd-tree and further accerlerated by inverted file/min-hashing (special LSH), reranked by geometric consistency;4) SIFT-based image/scene classification with SVMs/Naive Bayes classifiers.More details and results can be seen in my personal websites.

Product driven research in Media Networking Lab of Core-network R&D Dept. Led several projects as:1. Video retargeting (saliency-based DP); 2. Object-based video annotation tool (online learning-based tracking-as-detection) where object locations are saved in XML file;3. Video interaction (object detection, tracking and classification);4. Video summarization (story board & skimming);5. Public cloud (Amazon)-based video transcoding (load balancing and auto scaling);6. Video seamless ads insertion for augmented reality.Also led some collaboration projects with academic schools, follow as:1) Interactive contextual targeted video ads (video concept classification, ads categorization and ads-video scene matching) with Peking University China; 2) Sports player detection for video highlighting with U. of Missouri at Columbia;3) Merchandise classification for video annotation with State U. of NY at Buffalo.Part of work published at IEEE T-CASVT'12, ICIP'11, ICME'11 and ICIP'10. More details and demos see my personal website.

Research in Signal Acquisition & Processing team.1) Object detection, segmentation and tracking algorithms for project "object highlighting" (mobile video low bit rate compression with h.264).2) Mosaic generation from videos. Supervised postdoctor for research in mosaic-based figure-ground segmentation. 3) View synthesis with multiple depth images and depth estimation from rectified views for the new standard 3DV (extension of MVC).More demo videos are shown from my personal website. Meanwhile, published papers (ICPR'08, ICME'07, SIP'07, MCAM'07) about my research work are also posted.

Working in New Product Development group for the next generation explosive detection systems (EDS) .Explosive detection and classification based on X-ray images and scattered spectrum: segmentation, partial volume compensation, MLP-based training and testing.Volume data segmentation by watershed and volume rendering with the fast ray-casting algorithm, i.e. shear warp factorization.