Daniël De Kok Email & Phone Number

At Explosion I work on the Thinc machine learning library and the spaCy natural language processing library. Some of the things I worked on:- Designed and implemented Curated Transformers. Curated Transformers is a Python library of building blocks to implement transformer models. Curated Transformers supports popular encoder models such as XLM-RoBERTa, RoBERTa, BERT, and CamemBERT, as well as decoder models such as Llama 1/2, Falcon, GPT-NeoX, and MPT. Curated Transformers also provides greedy and sampling generators for LLMs. Curated Transformers will be used in spaCy 3.7 for transformer pipelines.- Implemented a new lemmatizer that replaces the rule-based lemmatizer for most pretrained models. The new lemmatizer uses edit trees that are predicted by a convolution or transformer model.- Implemented model distillation for extracting smaller models from larger teacher models.- Implemented a graph-based dependency parsing model using a biaffine model for arc predictions as an alternative to the transition-based parser of spaCy.- Added support for Metal Performance Shaders to spaCy and Thinc, making it possible to accelerate inference on Apple Silicon GPUs.- Made many CPU and GPU performance improvements across the board.

Groningen, Netherlands

- Further development of the SyntaxDot multi-task syntax annotator (in Rust), including a biaffine parser.- Customizations of SyntaxDot for customers, including a Java binding.- Developed an taught a course 'Transformers from Basic Principles'. This course took a bottom-up approach to teaching transformers, starting from basic linear algebra, finding derivatives, linear classifiers, working to full transformer networks step by step.

- My research focused on neural network models for syntax annotation, with a strong focus on dependency parsing, topological field annotation, and multi-task learning. This work culminated in several papers, including a paper selected as Outstanding Paper at ACL 2016.- Successfully applied for a renewal of a SFB (Sonderforschungsbereich) on composition models for phrases as a co-principal investigator, receiving funding for 2 PhD students and several work students.- Taught many courses in the International Studies in Computational Linguistics (ISCL) BA and MA programs. Including: advanced programming course (Java), Text Technology, Information Retrieval, Deep Learning, Parsing with Prolog, and an introductory Rust course.- Supervised several BA and MA theses.- For two years, I was the student advisor for the ISCL BA program.- Member of various committees, such as the accreditation renewal for the ISCL BA/MA programs, primarily focusing on renewing the BA program's exam regulations and module handbook.

Worked in the CLARIN-D project on:- Improving reliability of WebLicht web services.- Improving scalability of WebLicht web services, including parallelization of heavier services such as parsers.- Rewrote core parts if the TüNDRA treebank search application for treebanks for better performance and to give incremental results while a query is executed.- Implemented several new WebLicht web services.

Amsterdam Area, Netherlands

Worked on various parts of Gridline's language processing pipeline:- Implemented a new lemmatizer based on edit scripts and support vector machines (SVMs) to replace Gridline's rule-based lemmatizer.- Implemented a modular spell checking architecture.- Improvement of fuzzy lookups in various parts of Gridline's language processing pipeline using dictionary automata and Levenshtein automata.- Implemented support for dependency annotations in Gridline's pipeline using Alpino.- Added initial support for stonger typing of pipes using Apache UIMA.Besides my work on the language processing pipeline, I also worked in client projects, such as tailoring the Gridline ES search engine for specific customers.

My PhD research project focused on natural language generation in the Alpino system for Dutch. Together with Gertjan van Noord, I implemented a chart generator that uses the Alpino grammar. I also developed a fluency ranker using maximum entropy models. The end result of the project was a truly reversible model, where the same grammar and maximum entropy component could be used for parsing and generation.During my PhD I also taught the course Natuurlijke-taalverwerking 1 `Natural Language Processing 1`, which focused on implementing computational grammars and parsers in Prolog.I also developed various tools, such as the Dact treebank search application in C++ and Qt and a maximum entropy optimizer in C.

Developed a hidden Markov model part-of-speech tagger and a decision-list based lemmatizer in Java.

Created an online course for high-school students on information retrieval and natural language processing. After five years, this course is still used as-is by the university.Co-developed a content-management system for works of art, to support a joint research project of the University of Groningen, The Groningen Museum, and the Werkman Foundation. This project made an inventory of all the works, correspondences, and exhibitions of the Dutch painter H.N. Werkman.I was the teaching assistant for the XML and Natural Language Processing courses of the Information Science program.

Vancouver, Canada Area (Remote)

Co-developed the graphical installer and graphical configuration tool for Libranet Linux 3.0. Libranet was a commercial distribution based on Debian GNU/Linux, oriented at new Linux users. The installer and configuration tool were written in Perl using the Gtk2+ toolkit.

Doctor Of Philosophy - Phd, Computational Linguistics, Graduated With Distinction

Activities and Societies: Chair PhD association (Grasp)

Master Of Arts - Ma, Information Science/Studies, Graduated With Distinction

Bachelor Of Arts - Ba, Information Science/Studies, Graduated With Distinction

Propaedeutic Certificate, Philosophy