My current research focuses on fine-tuning multi-modal LLMs for high-resolution dense predictions, such as panoptic segmentation on 4-8MP images. Our primary goal is to improve panoptic segmentation accuracy while maintaining any prior MLLM capabilities, without fine-tuning on multi-modal datasets. Future work will focus on improving efficiency, demonstrating new capabilities and extending to new application areas. Previous work focused on applying weight and activation sparsity to improve… Show more My current research focuses on fine-tuning multi-modal LLMs for high-resolution dense predictions, such as panoptic segmentation on 4-8MP images. Our primary goal is to improve panoptic segmentation accuracy while maintaining any prior MLLM capabilities, without fine-tuning on multi-modal datasets. Future work will focus on improving efficiency, demonstrating new capabilities and extending to new application areas. Previous work focused on applying weight and activation sparsity to improve hardware efficiency, using the roofline model to understand GPU hardware limits. My anticipated graduation date is between December 2024 - March 2025. Show less

The first half of my PhD was focused on GPU computing techniques to maximize throughput on compute-intensive workloads. My main research project was designing better set-intersection algorithms to improve GPU throughput for social network applications. During this time I started learning about machine learning and computer vision, and decided to transition my PhD to these areas while using my GPU experience to identify areas to improve efficiency.

As a technical consultant for Stratovan my work involved a mix of project management, presentations, research and development for our airport checked baggage explosive detection pipeline. I worked with a team of software engineers to analyze the accuracy and latency of our object detection pipeline and plan software improvements. I presented our work to the Transportation Security Administration in Washington, DC and other aviation partners in the surrounding area. I also helped with data… Show more As a technical consultant for Stratovan my work involved a mix of project management, presentations, research and development for our airport checked baggage explosive detection pipeline. I worked with a team of software engineers to analyze the accuracy and latency of our object detection pipeline and plan software improvements. I presented our work to the Transportation Security Administration in Washington, DC and other aviation partners in the surrounding area. I also helped with data acquisition to ensure we acquired sufficient data for training our explosive detection models. Show less

As a machine learning software engineer my primary focus was on designing and implementing a scalable, high-performance object classification system for explosive detection in CT scans. Given a set of hand-designed features from an image segmentation and feature extraction stage, I used linear and non-linear machine learning models to classify each segmented object. The dataset was very small, from hundreds to thousands of ground truth objects, which required careful design and consideration to… Show more As a machine learning software engineer my primary focus was on designing and implementing a scalable, high-performance object classification system for explosive detection in CT scans. Given a set of hand-designed features from an image segmentation and feature extraction stage, I used linear and non-linear machine learning models to classify each segmented object. The dataset was very small, from hundreds to thousands of ground truth objects, which required careful design and consideration to avoid overfitting. I scaled the classification system to a multi-node multi-CPU cluster to reduce the optimization pipeline from multiple days to several hours. This allowed us to rapidly iterate over different designs and datasets for various aviation security partners. Show less

As a GPGPU software engineer my primary task was to implement and maintain an OpenCL wrapper to transition our image segmentation framework from CPUs to GPUs. I used profiling tools to identify bottlenecks in the codebase and replace OpenMP-based loops with OpenCL. We saw significant speed-ups across many stages using GPUs.