Kyle Dorman Email & Phone Number

- Outlined company’s year one ML roadmap. Detailed types of ML models necessary and existing relevant research papers, datasets, and open source projects to leverage- Built a cloud based machine learning inference pipeline to detect, track, and classify products and empty shelf areas of retail stores collected from human mounted cameras- Trained bounding box detectors, segmentation, and classification models using open datasets and in house collected data- Designed an auto labeling segmentation pipeline for background and dynamic objects using GroundingDino and Segment Anything- Prototyped semi-automatic item classification data labeling pipeline which leverages store employees scanning products while picking online grocery orders- Research different out of distribution classification techniques on a 1600 class, 200K image retail products dataset. Final model had 94% accuracy on in distribution classes and 80% recall on out of distribution classes

San Francisco, Ca, Us

- Refactored an existing PyTorch keypoint estimation model used to predict single object 6D poses (position and orientation). Resulted in significant improvements to the model's speed and accuracy- Improved robustness of keypoint estimation model trained only on synthetic data by adding image augmentations which helped close the gap between synthetic and real world performance- Implemented process and tooling to collect ground truth data for evaluating accuracy of multi-view 3D object localization models using OpenCV and Streamlit- Designed a multi-view reprojection based 6D pose refinement algorithm using OpenCV and Scipy for use with a deep learning based keypoint prediction model. Produced translation errors of less than 3 millimeters - Created a multi-view, multi-fiducial shift estimation algorithm with less than 1 millimeter translation error and 1 degree rotation error using AprilTags, OpenCV, and Scipy- Researched intrinsic camera calibration requirements and prototyped an interactive UI to ensure quality calibrations from non-expert staff in OpenCV and Streamlit- Built an end to end Pick and Place demo using a robotic arm, pattern projector camera, Halcon, Open3D, and Jupyter- Evaluated and developed robot hand-eye calibration methods for pattern projector cameras with OpenCV, Open3D, and Scipy

San Francisco, California, Us

Human Keypoint Detection- Architected a flexible human keypoint estimation deep learning training code base with Tensorflow which enabled team members to easily iterate on model architectures and model outputs- Retrained OpenPose model with mixed precision for 2x inference speed with no loss in accuracy- Designed a new pose model based on latest research papers with custom post processing algorithms which reduced false positive key point association rate and increased the main accuracy metric (OKS) by 10 points- Implemented end to end data collection, hard example mining, data labeling and model retraining workflow. Model’s accuracy in visually distinct stores increases 10-20 points after retraining with store specific dataAdditional Models & System Architecture- Architected a new version of the real time ML pipeline to run in GCP. Lead a team of four engineers to switch to a new camera model, change video acquisition methods, rewrite the per frame ML service in gStreamer to run on T4 Nvidia cards and build a new stand alone tracking service in Rust. System allows for simpler on site installation, is more fault tolerant, and simplifies team’s service ownership- Integrated new pose model into a real-time multi-camera tracking algorithm in Rust. New pose model and tracking algorithm resulted in 50% less identity swaps on the test set- Refactored Rust tracking algorithm to enable experimentation with different internal algorithms of the overall tracking system- Designed an ML pipeline which localizes and classifies digits on smartphones from overhead cameras using python, OpenCV, and Keras. System does not require real data for training, only for hyper param tuning and accuracy evaluation- Contributed to a Tensorflow per store item classification model and a multi-camera event prediction model

Boston, Massachusetts, Us

- Spent time on frontend team, iOS mobile team, and backend team- Overhauled legacy Objective-C iOS app by porting pieces of the app to Swift as a member of a four-person team- Migrated legacy desktop web apps to responsive web apps on a team of three- Administered Software Engineering Summer Apprentice Program. Led recruiting, interviewing and project planning

Boston, Ma, Us