Principal Software Engineer
CurrentWorking on a solution (Java) that allows users to train / deploy ML models for data extraction (models are python / torch based, delivered as Docker images, managed via a Kubernetes backend).Lots of troubleshooting leading to fixes and improvements to the solution.Some notable activities: discovered (and fixed) improper use of green threads in pyhton code leading to abnormally large request duration under high load, implemented a tool-set to profile models in prod-like environments, decisive contributions to fixing CVEs, discovered a race-condition in torch-vision that prevented concurrent running of predicts, improved model speed by page-level parallelism, improved GPU utilization (and ultimately speed) by fixing data pipeline bottlenecks, found and fixed source of high memory consumption in data pipeline.As POCs: re-architected model to achieve maximum GPU utilization, re-architected model to allow serving multiple user models on the same system (loading up to 20 user models per GPU, also allowing hot-swapping).Special focus was on the data pipeline optimization via re-factoring and by leveraging multi-processing, leading to improved GPU utilization both at inference and at training time.