Junior Data Scientist
CurrentAs a Data Scientist in my current role, Spearheaded the development of a search and chatbot system leveraging Elasticsearch and LLMs to streamline document retrieval and improve user interaction, enhancing operational efficiency by 30%. Automated the ingestion of large volumes of documents from AWS S3 using PDFPlumber, PyMuPDF, and AWS Textract, reducing manual handling by 80%. Developed complex algorithms to convert documents into embeddings and implement cosine similarity and KNN for accurate document ranking in Elasticsearch. Implemented a Retrieval-Augmented Generation (RAG) system and multi-threading to optimize search performance, reducing query response times. Integrated LLMs for generating precise responses to user queries, improving engagement by 25%. Enhanced the chatbot's memory to retain context, improving accuracy in multi-step user queries. Built real-time communication capabilities through socket programming, delivering a responsive user experience. Processed form data using computer vision techniques like SIFT and fuzzy matching to convert hardcopy forms into digital formats and generate actionable insights. Led the creation of dynamic data visualizations to empower users with in-depth analysis and comparative insights.