Research Engineer
Meta Superintelligence Labs
prajj at meta dot com

Bio

I’m Praj, I work as an AI Researcher at Meta Superintelligence Labs working on building foundational models. I am a core contributor of LLaMA 4, LLaMA 3 LLaMA 2, LLaMA 2 Long, powering Meta’s flagship AI assistant meta.ai. My primary focus has been on fundamental pre-training research and infra around it. One of my primary contributions include building long context capabilites for LLaMA 4 both on modeling and pre-training infra / inference. LLaMA 4 can attend to documents exceeding 10M tokens. Previously I worked as an AI Resident within Reality Labs and Fundamental AI Research (FAIR) at Meta working on Offline Reinforcement Learning. My google scholar can be found here.

Prior to Meta, I was a CS graduate student at the University of Texas Dallas where I worked on commonsense reasoning under the supervision of Prof. Vincent Ng. My thesis is about improving commonsense reasoning through adversarial learning.

Publications

BTS: Harmonizing Specialized Experts into a Generalist LLM

Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis
Paper

The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation

Generative AI, Meta
Paper

The LLaMA 3 herd of models

Generative AI, Meta
Paper

Correlating and Predicting Human Evaluations of Language Models from NLP Benchmarks

Rylan Schaeffer, Punit Singh Koura, Binh Tang, Ranjan Subramanian, Aaditya K Singh, Todor Mihaylov, Prajjwal Bhargava, Lovish Madaan, Niladri S. Chatterji, Vedanuj Goswami, Sergey Edunov, Dieuwke Hupkes, Sanmi Koyejo, Sharan Narang
Paper

Effective Long-Context Scaling of Foundation Models

W. Xiong, J. Liu, I. Molybog, H. Zhang, P. Bhargava , R. Hou, L. Martin, R. Rungta, K. Sankararaman, B. Oguz, M. Khabsa, H. Fang, Y. Mehdad, S. Narang, K. Malik, A. Fan, S. Bhosale, S. Edunov, M. Lewis, S. Wang, H. Ma
Paper

Llama 2: Open Foundation and Fine-Tuned Chat Models

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava , S. Bhosale, D. Bikel, L. Blecher, C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. Smith, R. Subramanian, X. Tan, B. Tang, R. Taylor, A. Williams, J. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, T. Scialom
Paper Official Announcement Code

Sequence Modeling is a Robust Contender for Offline Reinforcement Learning

Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang
International Conference on Learning Representations (ICLR) 2024 arXiv Code Bibtex

AUTODIAL: Efficient Asynchronous Task-Oriented Dialogue Model

Prajjwal Bhargava, Pooyan Amini, Shahin Shayandeh, Chinnadhurai Sankar
arXiv Code Bibtex

DiscoSense: Commonsense Reasoning with Discourse Relations

Prajjwal Bhargava and Vincent Ng
EMNLP 2022 arXiv Code Bibtex

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Prajjwal Bhargava and Vincent Ng
AAAI 2022 Paper Poster Bibtex

Generalization in NLI: Ways to [Not] Go Beyond Simple Heuristics

Prajjwal Bhargava, Aleksander Drozd, Anna Rogers
EMNLP Workshop on Insights from Negative Results 2022 Paper Code (Huggingface) Code (Pytorch Lightning) Bibtex Presentation video Poster Slides

Adaptive Transformers for Learning Multimodal Representations

Prajjwal Bhargava
ACL SRW 2022 Paper Code Bibtex Presentation Video

On Generalization of Detection Models for Unconstrained Environments

Prajjwal Bhargava
ICCV AutoNUE Workshop 2022 Paper Code Bibtex Poster

Incremental Learning in Person Re-Identification

Prajjwal Bhargava
arXiv preprint Paper Code Bibtex Poster

Side projects