Introduced LLMs in text postprocessing and fact extraction tasks. Contributed to NeoML (PyTorch analogue), enabling it to run BPE and Transformer Encoder.Pretrained a LSTM-like language model and used it for normalization task, which made field extraction fill in data in the nominative case.First who delivered BERT / RoBERTa into production at ABBYY. Solved the semantic segmentation task using RoBERTa, LoRA and proper scheduling, achieving an increase in quality from 70% to 97% on a scientific dataset and avg. 3% on all datasets.Created a DeBERTa-based NER / fact extraction solution that halved the error rate compared with the existing solution.NeoML. Made BPE implementation 20 times faster, added the Unigram algorithm. Fixed numerous bugs in the Transformer implementation, making it 100% compatible with PyTorch. Tested LoRA and Python (pybind11) wrappers. Participated in code reviews.Created a rough analogue of PyTorch Lightning for NeoML, enabling the team to train and infer any network on different devices (including DDP) using the one simple interface. Implemented optimizations that can speed up the inference on CPU by 8-10 times (depending on data and hardware).Pretrained static embeddings with BPE dictionary solving OOV problem at the request from other teams.

Completely reworked an instrument to compare and debug ML solutions. Researched language models for OCR text refinement. Improved internal libraries and pipelines.The instrument compared two or three tree-like document markups with a known structure, calculated metrics, generated reports, and sent alerts. It was challenging to correctly match different objects while maintaining the tree structure and ontology restrictions. It also contained tangled logics with tons of different metrics prioritizing various client requirements.During refactoring, we successfully separated the monolith into matching, evaluation and report modules, which later allowed us to easily add new metrics and update the data format. Developed appropriate heuristics, applied fuzzy-matching and tree search. Improved error explanations and visualizations by creating a Vue.js web-application.Created a SAX interface for the internal C++ JSON-library. Reworked DOM interface making it 30% faster, more convenient and C++11+ compatible, reduced its memory consumption by 10-15 times.Participated in the design of a new company-wide markup format that replaced dozens of local standards. Wrote its C++ (30%) and Python (100%) implementations.Investigated the possibility of OCR postprocessing with pretrained language models. Despite being able to fix a significant amount of recognition errors, models of 2018-2020 years were either too small and made a minor contributuion to the already decent solution, or too slow.

Conducted algorithms seminars, created Deep Learning course and Efficient Deep Learning course. Mentored students theses.Algoritms: sorting, trees, stringsDL Course: pytorch, backprop, CNN, RNN, Transformer, LLMEfficient DL Course: AMP, ONNX, LoRA, quantization, tensorboard