NLP developer working in the enhancement and development of applications based on Large Language Models (LLMs).Some of the projects I have worked on:Performance Evaluation of Rankers and RRF Techniques for Retrieval Pipelines:I have done a comparative study of adding different combinations of rankers in a Retrieval pipeline along with the use of Reciprocal Rank Fusion (RRF) techniques. The results were evaluated on four metrics: Normalized Discounted Cumulative Gain (NDCG), Mean Average Precision (MAP), Recall and Precision. We analyzed the effectiveness of adding different rankers to pipelines to improve the quality of retrieved documents.A comparison of Hyperparameter Tuning and Optimizer Selection on Training Efficiency and LLM Performance:Built a Question Answering system for News Articles. For the SQuAD dataset, a baseline was set using the BERT model. The BERT, DistilBERT and FinBERT models were fine-tuned using an AdamW optimizer.Built a sentiment analysis model to predict the sentiment of a Financial News article. For the Financial PhraseBank dataset, a baseline was set using the BERT model. The BERT, DistilBERT and RoBERTa models were fine-tuned using an AdamW optimizer.Built a summarization model that gives short and concise summaries for News Articles. For the Multi-News dataset, a baseline was set using the BART model. The BART, DistilBART models were fine-tuned using an AdamW optimizer.The optimizers were tuned for different parameters by a specific way of search spaces. Each parameter was first tuned on a large search space, and then a smaller and more refined search space was used to find the optimal value for training the model.A comparative study of different optimizers used for training was done. The optimizers were tuned for different parameters by a specific way of search spaces. Each parameter was first tuned on a large search space, and then a smaller and more refined search space was used to find the optimal value for training the model.Financial Dashboard : Built an end-to-end Financial Dashboard that collects and consolidates all of a business's critical observations in one place using the information obtained from the annual 10-K SEC Filings. It contains:- Insights and summaries for different sections from annual corporate filings.- Sentiment-based score that measures the company's performance over a certain time period.- Identification of Important topics and Frequently occurring words mentioned in the report.Email: varunm500@gmail.comGithub: https://github.com/vrunm
Open Source
-
Ai DeveloperOpen SourcePune, Mh, In
-
Open Source DeveloperOpen Source Aug 2023 - PresentWork done in Haystack:• Added the support for Cohere embedding models.• Added the support for INSTRUCTOR embedding models.• Created an integration for Pinecone Vector Database.• Added the support for LostInTheMiddleRanker for reranking tasks.• Added support for evaluation metrics like F1score, Exact Match and Semantic Answer Similarity.• Implemented the LLM-Blender as an ensembling framework designed to achieve consistently superior performance by combining the outputs of multiple language models (LLMs) in Haystack. Github: https://github.com/deepset-ai/haystack/pulls?q=is%3Apr+is%3Aclosed+vrunm
-
Ai Research Intern (Llms)Alphasense Feb 2023 - Jul 2023• Built a pipeline to perform open-ended question-answering on earnings call transcripts using Large Language Models (LLMs). The pipeline can answer questions on information present directly in a transcript, as well as combine information from multiple transcripts to answer indirect questions.• The pipeline uses Retrieval Augmented Generation (RAG) to incorporate new information, and does not require retraining the Generative LLMs. RAG retrieves data from outside the model, and augments the prompts by adding the retrieved data in-context. • The pipeline pre-processes the transcripts, and text snippets are extracted from each section of the earnings call. The text snippets are chunked dynamically based on text similarity. • The pipeline creates an embedding for each chunk, and stores these embeddings in a vector database. We experimented with the following SOTA embedding models: SBERT, MPNET, SGPT and INSTRUCTOR. The INSTRUCTOR model generated embeddings which resulted in the best context retrieval.• The embeddings are retrieved from the Vector Database, and used as context to the open-ended questions. The context is passed along with the question to the generative LLM for generating an accurate and concise answer to the question.• We compared different strategies for context retrieval, including dense embedding retrieval and hybrid retrieval. A combination of both strategies gave the best results. We created different prompt templates for entity extraction and question-answering. Weak supervision techniques were used to improve the prompts for extracting the entities and dynamically generating few-shot examples.• Carried out extensive prompt tuning by iteratively refining prompt formatting, instructions and incorporating few-shot examples.We experimented with the following instruction-tuned generative LLMs for generating answers: Llama-2, Vicuna, Alpaca, Dolly, FLAN-T5 and GPT-3. The Llama-2 and GPT-3 LLMs generated the most accurate and concise answers.
Varun Mathur Education Details
-
Cgpa 9.53 (Ranked 2ⁿᵈ In Batch) -
Cgpa 8.53
Frequently Asked Questions about Varun Mathur
What company does Varun Mathur work for?
Varun Mathur works for Open Source
What is Varun Mathur's role at the current company?
Varun Mathur's current role is AI Developer.
What schools did Varun Mathur attend?
Varun Mathur attended Fergusson College, Fergusson College, Delhi Public School, Pune.
Not the Varun Mathur you were looking for?
-
Varun Mathur
Strategic Business Leader | Driving Market Expansion, Operational Excellence & High-Impact Partnerships For Sustainable Growth | Iim AhmedabadNoida -
Varun Mathur
Engineering Manager At Flipkart | Delivering Cost Savings, Scaling Services | Ex-Microsoft, Ex-Cleartax | Iim Bangalure AlumniBengaluru -
2gmail.com, hdfcbank.com
-
2vccircle.com, pearson.com
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial