Undergraduate Teaching Assistant for the course MTL103: Optimization and Applications, Spring 2023.

Accelerating Transformer Compression with Variational Information Bottleneck based Pruning[Efficient Systems for Foundational Models (ICML) 2024]Advisor: Prof Sumeet Agarwal• Devised a variational information bottleneck-pruning method for the systematic compression of structural modules with user-defined constraints• Proposed 'fast' and 'faster' approach for ~25x faster compression, allowing the method to scale for larger models• Applied the methods on BERT, ROBERTa, GPT2 and evaluated the models on GLUE and SQuAD benchmarks• Adapted the faster approach to LLaMA-1/2/3 and Mistral-7B, achieving SoTA performance on reasoning tasks• Improved performance and inference via post-pruning weight update and dimension adaption for GPU parallelism

Undergraduate Teaching Assistant for the course MTL106: Probability and Stochastic Processes, Fall 2022.

Worked on data valuation methods taking into account the temporal dynamics of data and the inter-relatedness of data within time

Worked on a learning framework for neural architecture search modeling the human learning technique of Learning from Mistakes using data re-weighting and learner-reviewer models