Lead Data Scientist
Current**Personalized Test Simulation Solution for Edtech**- Led and developed an AI-driven adaptive learning system that personalizes question complexity based on individual performance, enhancing assessment accuracy and learner engagement. - Implemented a scalable solution supporting over 8000 skills, trained on 400,000 questions and ~2M user attempts. Designed a Change Data Capture (CDC) pipeline triggered by data changes, utilizing Celery, FastAPI, and Redis for data preprocessing and training. - Post-training, used an embedding model for clustering similar questions and monitored tasks with MongoDB and Sentry. - Deployed using GitHub CI/CD pipelines, managing staging and production environments. Technologies used: FastAPI, Celery, Redis, MongoDB, Sentry, GitHub CI/CD, Transformers.**Semantic Similarity Solution for Market Intelligence & Business Information**- Led a team in developing advanced semantic search engine solutions for fetching relevant companies, integrating FAISS-based similarity search with hierarchical clustering and topic modeling. - Deployed two variations: text-to-ElasticSearch for enhanced semantic understanding and industry taxonomy ontology for precise vector similarity search. - Designed and built the solution using FAISS, ElasticSearch, and Gensim, with FastAPI for robust querying. - Achieved a search latency of ~200ms on 400,000 company entries, significantly improving search relevance and accuracy. Technologies used: Python, ElasticSearch, FAISS, Gensim, FastAPI, Docker, GitHub, GitHub Actions (CI/CD).**Saathi-Catalyzing Equitable Artificial Intelligence (AI) Use**- Led the design and technical development of Saathi Chatbot(https://finsaathi.mgmt.iisc.ac.in/) in collaboration with IISc and Oxford Brookes University.- Bill and Melinda Gates Foundation, July 2023 – March 2024, Global Challenges Call