Publications
2022
Patching open-vocabulary models by interpolating weights
Gabriel Ilharco*, Mitchell Wortsman*, Samir Yitzhak Gadre*, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, Ludwig Schmidt
Preprint
[Paper] / [Slides]Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, Ludwig Schmidt
International Conference on Machine Learning (ICML) 2022
[Paper]Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)
Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, Ludwig Schmidt International Conference on Machine Learning (ICML) 2022
[Paper]CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration
Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, Shuran Song
[Paper]Robust fine-tuning of zero-shot models
Mitchell Wortsman*, Gabriel Ilharco*, Mike Li, Jong Wook Kim, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, Ludwig Schmidt
Conference on Computer Vision and Pattern Recognition (CVPR) 2022
[Paper] / [Code]
2021
Probing contextual language models for common ground with visual representations
Gabriel Ilharco, Rowan Zellers, Ali Farhadi, Hannaneh Hajishirzi
North American Chapter of the Association for Computational Linguistics (NAACL) 2021
[Paper] / [Slides]Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Jesse Dodge, Maarten Sap, Ana Marasovic, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner
Empirical Methods in Natural Language Processing (EMNLP) 2021
[Paper] / [Code]MultiModalQA: complex question answering over text, tables and images
Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant
International Conference on Learning Representations (ICLR), 2021.
[Paper] / [OpenReview]Finetuning Pretrained Transformers into RNNs
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A Smith
Empirical Methods in Natural Language Processing (EMNLP) 2021
[Paper]Contrasting Contrastive Self-Supervised Representation Learning Pipelines
Klemen Kotar, Gabriel Ilharco, Ludwig Schmidt, Kiana Ehsani, Roozbeh Mottaghi
International Conference on Computer Vision (ICCV) 2021
[Paper]
2020
Evaluating models’ local decision boundaries via contrast sets
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
Conference on Empirical Methods in Natural Language Processing (EMNLP) Findings 2020
[Paper]Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, Noah Smith
[Paper]Toward ML-centric cloud platforms
Ricardo Bianchini, Marcus Fontoura, Eli Cortez, Anand Bonde, Alexandre Muzio, Ana-Maria Constantin, Thomas Moscibroda, Gabriel Ilharco, Girish Bablani, Mark Russinovich
Communications of the ACM 2020
[Paper]
2019
Large-scale representation learning from visually grounded untranscribed speech
Gabriel Ilharco, Yuan Zhang, Jason Baldridge
Conference on Computational Natural Language Learning (CoNLL) 2019
[Paper] / [Code]General evaluation for instruction conditioned navigation using dynamic time warping
Gabriel Ilharco, Vihan Jain, Alexander Ku, Eugene Ie, Jason Baldridge
Visually Grounded Interaction and Language Workshop at NeurIPS 2019
[Paper] / [Code]Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Vihan Jain*, Gabriel Ilharco*, Alex Ku*, Ashish Vaswani, Eugene Ie, Jason Baldridge
Annual Meeting of the Association for Computational Linguistics (ACL) 2019
[Paper] / [Slides] / [Code]Transferable representation learning in vision-and-language-navigation
Haoshuo Huang, Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Ilharco, Jason Baldridge, Eugene Ie
International Conference on Computer Vision (ICCV) 2019
[Paper]