Computational Linguistic Developer
Current•Implemented a high-performance word decompounder utilizing Google's Trillion Word Corpus.•Developed a hybrid solution combining Peter Norvig’s statistical methods and word embeddings to accurately split compound words while preserving original formatting.•Tasks included implementation of word merging algorithms, segmentation logic, and compound word detection, ensuring compatibility with the existing test cases.•Optimized preprocessing pipeline and caching mechanisms to efficiently handle millions of records with complex social media text and hashtags.