Working on GenAI R2P Platform. My org is responsible for bringing research from Llama side to internal Meta teams and provide platform for all GenAI initiatives in the company.

Building AI Infrastructure. TL for revamping Recommendation systems model training stack for IGML. Worked across several teams on both sides : AI Infra and IGML Infra to make this happen

Worked in the Data Infrastructure group mainly focussed on distributed data processing (Hive, Hadoop, Spark) and resource management.Co-founded Spark team @ Meta and scaled it from 3 -> 35 engg. TL for platform reliability and SQL layer. Some of my work :- https://code.facebook.com/posts/678403995666478/using-apache-spark-for-large-scale-language-model-training/- https://www.youtube.com/watch?v=6BD-Vv-ViBw- https://www.youtube.com/watch?v=qjW9iJ70DSM- https://www.youtube.com/watch?v=ur_wLGE1aGA

My work has been motivated from scaling Apache Spark to large scale production deployments at Facebook. This was in form of making performance improvements, adding features, ensuring reliability and bug fixes. A good chunk of that is in the SQL space.

Contributing to the open source project, Apache Nutch, by fixing bugs and helping people over user group. I am mainly focused for improvements with the crawling phases (inject, generate, fetch, update). Notable features / Jiras I have worked on:-------------------------------------------------------------Support sitemaps in Nutch : https://issues.apache.org/jira/browse/NUTCH-1465HostDB for Nutch : https://issues.apache.org/jira/browse/NUTCH-1325Make Injector a single mapreduce job : https://issues.apache.org/jira/browse/NUTCH-1712

Worked on Apache Kafka, the distributed pub-sub messaging system developed at Linkedin. My project was a new feature for Apache Kafka: Inbuilt Offset Management for Kafka Consumers. This feature would remove the dependency on Zookeeper for offset storage and would store the conusmer offsets in Kafka itself. Here is a wiki of the implementation details:https://cwiki.apache.org/confluence/display/KAFKA/Inbuilt+Consumer+Offset+ManagementApache Kafka Jira:https://issues.apache.org/jira/browse/KAFKA-1012

• Worked on design and implementation of an enterprise search engine.• Complete ownership of the crawler module. Enhanced the Apache Nutch crawler as per design requirements, added features to it, fixed bugs and managed build and deliveries. • Developed analytics for web-data in IBMs’ JAQL (Java Annotated Query Language)• Used HBase for data storage. Knowledge about Cassandra and basic querying using Pig.• Backend cluster setup and management. Worked on Amazon EC2.• Direct interaction with customers and clients (both written and verbal communication). • Got appreciation from customers, clients and peers for good quality deliverables and intergrity.

• Worked on design and implementation of an enterprise search engine.• Enhanced the Apache Nutch crawler as per design requirements, added features to it, fixed bugs.• Backend cluster setup and management.• Direct interaction with customers and clients (both written and verbal communication). • Awarded “You Made a Difference” for my contribution (within 1 year after I joined).

Worked as a project trainee in Software Engineering division at TRDDC. Work involved developing a algorithm for a given problem and creating a tool to depict it. More details on the project "Generic Semantic Program Differencing Tool" in the projects' section.