– Contributing to the development of products leveraging LLM based agents.– Built a multi-agent system capable of handling tasks such as analyzing large datasets (structured and unstructured), interacting with various platforms, all driven through natural language queries. Agents seamlessly integrate with internal systems to process, analyze, and provide insights from diverse data sources.

Supervisors: Dr. Kalika Bali, Dr. Sunayana Sitaram– Sanmati∗ Research focuses on measuring and mitigating gender biases in datasets and LLMs.∗ We investigated the generation of gender biased sentences from GPT-3.5-Turbo and the consequentreceptivity of human annotators to bias of varying degree (paper accepted at EMNLP'23).∗ Currently focusing on adding cultural nuances to GPT generations, crowd-sourced generation of biasedsentences via gamification, and mitigation of gender bias in generative models via preference modeling.– Shiksha Co-pilot∗ Contributed to the development of a product aimed at assisting teachers and intermediaries ineducation.∗ Responsible for end-to-end RAI harms measurement and mitigation before product release.∗ Developed a comprehensive framework to measure various harms associated with the product.∗ Mitigated the potential risks via careful prompt design and product changes.∗ Product successfully passed deployment safety and AI sensitive use review.– METAL: Towards Multi-lingual Meta Evaluation∗ Research focused on the use of LLM-based evaluators to help scale up multilingual evaluation.∗ In collaboration with the MS Word Co-pilot team, we investigated the capabilities of GPT-4 as anevaluator in text generation tasks across multiple languages (paper accepted at findings of EACL'24) ∗ Developed the first framework for end-to-end evaluation of LLMs as evaluators in multi-lingualscenarios.– MEGA: Multi-lingual Evaluation of Generative AI models∗ Research focused on evaluating generative AI models on standard NLP tasks in various languages.∗ Designed prompts and evaluated models on Question-Answering tasks and RAI tasks such as toxicityand gender bias in translations (paper accepted at EMNLP'23)

Supervisor: Dr. Nava Tintarev– Measuring Viewpoint Diversity of Conversations on Twitter.∗ Conducted research funded by Twitter, focused on measuring the health of social media conversationsand topics through various indicators.∗ Developed 2 metrics (Representation and Fragmentation) to quantify viewpoint diversity inconversations (paper accepted at WSDM'23)

Supervisors: Dr. Ekaterina Shutova (ILLC, UvA), Dr. Saif Mohammad (NRC, Canada)– Ruddit: Norms of Offensiveness in English Reddit Comments∗ Research funded by the Facebook Online Safety Benchmark Research Award to study the use ofoffensive language on social media platforms.∗ Created a novel dataset of social media comments with fine-grained degree of offensiveness scores usinga comparative annotation setup. The dataset was carefully created such that it avoids inappropriatebiases present in existing datasets (paper accepted at ACL'21)

Teaching assistant for Natural Language Processing 1 course for the Master's in Artificial Intelligence programme

Supervisor: Dr. Jelte Mense– Relevance Prediction of Security-Related Articles for The Dutch National Police.∗ Project focused on developing AI models to provide early threat alerts and actionable information forthe Dutch National Police.∗ Created a dataset of news articles using the GDELT 2.0 dataset and built computational models withhigh performance values.

Worked on re-engineering an expert system shell (Medium sized AI software)