Denis Boyda, Ph.D. Email & Phone Number

San Francisco Bay Area

* Enhanced fault-tolerant allreduce bandwidth by 2x through CUDA kernel optimization.

• Lead a team of 5 researchers and collaborate with 4 stakeholders to scale a prototype approach for production-level particle physics simulations on > 25k GPUs at supercomputer Aurora.• Optimize AI models though developing custom CUDA/SYCL kernels.• Developed tensor parallelism ML model scaling on 1024 Nvidia A100.

• Achieved a 75% performance increase in a state-of-the-art machine learning model for high-energy physics simulations by addressing a critical architectural issue and proposing physics-inspired features.• Mentored a student who was named one of 300 semifinalists in the Regeneron Science Talent Search.

Иллинойс, Соединенные Штаты Америки

• Collaborated with numerous interdisciplinary teams by profiling and optimizing their machine learning models for scientific applications on Nvidia GPUs (up to 2240 Nvidia A100), covering areas such as physics simulations, LLMs in biology, and hyperparameter optimization.• Enhanced the performance of machine learning-based particle physics simulations by 2x through migrating CPU operations to GPU, developing custom GPU kernels, utilizing operation fusion with nvFuser, applying CUDA graph compilation, and implementing advanced algorithmic solutions.• Increased the scaling efficiency of AI models by 60% on the Polaris supercomputer (2240 A100 GPUs) by implementing optimal NUMA binding and fine-tuning NCCL and DDP settings.• Mentored hackathon teams and led tutorials and hands-on sessions on performance optimization for deep learning frameworks.• Established, deployed, and managed MLOps (wandb, Sacred+mongoDB, AIM) and CI/CD pipelines.• Worked with stakeholders on the development of Deep Learning frameworks for supercomputer Aurora, identifying critical aspects of operations coverage, proposing milestones, and introducing benchmarks.

• Proposed and developed an ML-based approach for particle physics simulations, creating PyTorch scripts with custom autograd functions, custom kernels, and complex numbers (prior to PyTorch v1.6). Optimized memory allocation to fit models on a single GPU. The proposed approach demonstrated 1000x improvement in benchmarking problems.• Supervised five Undergraduate Research Opportunities Program (UROP) students.

• Developed a software package for simulating electromagnetic properties in condensed matter physics systems. The package, written in C++ with CUDA kernels for sparse matrix multiplications, utilized MPI for inter-worker communication and coordination. Successfully ran simulations on a machine equipped with 40 Nvidia GPUs.

• Developed a software package for high-energy physics simulations, written in C++ with CUDA kernels for physical operators. Employed MPI for result analysis and inter-process communication. Successfully deployed and ran the package on a local cluster equipped with 10 Nvidia GPUs.• Refactored legacy code and developed a package for biophysics simulations, incorporating MPI to scale simulations across several hundred CPUs.

Doctor Of Philosophy - Phd, Theoretical Physics, Gpa N/A - Grades Not Given

Activities and Societies: • The Kurchatov Institute is a leading Russian institute in the field of nuclear energy. • Earned the Russian.

Master Of Computer Applications - Mca, It, Gpa 5.00/5.00

Master Of Science - Ms, Theoretical Physics, Gpa 5.00/5.00

Activities and Societies: • Earned Governor's scholarship • Served as President of the University Tourism Club.

Bachelor'S Degree, Physics, Gpa 5.00/5.00

Activities and Societies: • Organized and served as the first president of the Student Scientific Society at the Institute of Physics.