ARC Baselines

We also provide neural baselines that we have run against ARC. The execution framework (also provided) is easily extensible to test new models on the ARC Question Set.

  • DecompAttn, based on the Decomposable Attention model of Parikh et al. (2016), a top performer on the SNLI dataset.
  • BiDAF, based on the Bidirectional Attention Flow model of Seo et al. (2017), a top performer on the SQuAD dataset.
  • DGEM, based on the Decomposable Graph Entailment Model of Khot et al. (2018), a top performer on the SciTail dataset.
  • Knowledge-free BiLSTM Max-out model with max-attention from question to choices, adapted by Mihaylov et al. from the model of Conneau et al. (2017).

Details of the first three systems and their adaptation for the multiple choice setting are given in (Clark et al. 2018). The BiLSTM Max-out model is described in this README.

The models can be downloaded here.