Publications

2022

• Patching open-vocabulary models by interpolating weights
  Gabriel Ilharco*, Mitchell Wortsman*, Samir Yitzhak Gadre*, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, Ludwig Schmidt
  Preprint
  [Paper] / [Slides]

• Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
  Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, Ludwig Schmidt
  International Conference on Machine Learning (ICML) 2022
  [Paper]

• Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)
  Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, Ludwig Schmidt International Conference on Machine Learning (ICML) 2022
  [Paper]

• CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration
  Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, Shuran Song
  [Paper]

• Robust fine-tuning of zero-shot models
  Mitchell Wortsman*, Gabriel Ilharco*, Mike Li, Jong Wook Kim, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, Ludwig Schmidt
  Conference on Computer Vision and Pattern Recognition (CVPR) 2022
  [Paper] / [Code]

2021

• Probing contextual language models for common ground with visual representations
  Gabriel Ilharco, Rowan Zellers, Ali Farhadi, Hannaneh Hajishirzi
  North American Chapter of the Association for Computational Linguistics (NAACL) 2021
  [Paper] / [Slides]

• Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
  Jesse Dodge, Maarten Sap, Ana Marasovic, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner
  Empirical Methods in Natural Language Processing (EMNLP) 2021
  [Paper] / [Code]

• MultiModalQA: complex question answering over text, tables and images
  Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant
  International Conference on Learning Representations (ICLR), 2021.
  [Paper] / [OpenReview]

• Finetuning Pretrained Transformers into RNNs
  Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A Smith
  Empirical Methods in Natural Language Processing (EMNLP) 2021
  [Paper]

• Contrasting Contrastive Self-Supervised Representation Learning Pipelines
  Klemen Kotar, Gabriel Ilharco, Ludwig Schmidt, Kiana Ehsani, Roozbeh Mottaghi
  International Conference on Computer Vision (ICCV) 2021
  [Paper]

2020

• Evaluating models’ local decision boundaries via contrast sets
  Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
  Conference on Empirical Methods in Natural Language Processing (EMNLP) Findings 2020
  [Paper]

• Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
  Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, Noah Smith
  [Paper]

• Toward ML-centric cloud platforms
  Ricardo Bianchini, Marcus Fontoura, Eli Cortez, Anand Bonde, Alexandre Muzio, Ana-Maria Constantin, Thomas Moscibroda, Gabriel Ilharco, Girish Bablani, Mark Russinovich
  Communications of the ACM 2020
  [Paper]

2019

• Large-scale representation learning from visually grounded untranscribed speech
  Gabriel Ilharco, Yuan Zhang, Jason Baldridge
  Conference on Computational Natural Language Learning (CoNLL) 2019
  [Paper] / [Code]

• General evaluation for instruction conditioned navigation using dynamic time warping
  Gabriel Ilharco, Vihan Jain, Alexander Ku, Eugene Ie, Jason Baldridge
  Visually Grounded Interaction and Language Workshop at NeurIPS 2019
  [Paper] / [Code]

• Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
  Vihan Jain*, Gabriel Ilharco*, Alex Ku*, Ashish Vaswani, Eugene Ie, Jason Baldridge
  Annual Meeting of the Association for Computational Linguistics (ACL) 2019
  [Paper] / [Slides] / [Code]

• Transferable representation learning in vision-and-language-navigation
  Haoshuo Huang, Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Ilharco, Jason Baldridge, Eugene Ie
  International Conference on Computer Vision (ICCV) 2019
  [Paper]