09-16-2021-jam.md

• SITUATEDQA: Incorporating Extra-Linguistic Contexts into QA, Michael J.Q. Zhang et al. emnlp2021 qa analysis

  • They propose a qa setting named SituatedQA, where (temporal, geographical) contexts influece the answer.
  • Focusing on information seeking questions
  • They identify situated qa pairs in existing qa datasets (16% of NQ-Open dataset)
  • How to create context? crowdsourcing for (context, answer) w.r.t. such questions
  • Quantify how traditional models fail on situated questions
  • My two cents This paper demonstrates a way of doing research, that is to find out or create in/from test sets new instances which cannot be handled well by current model

• The Grammar-Learning Trajectories of Neural Language Models, Leshem Choshen et al. analysis learning dynamics

• Cross-Task Generalization via Natural Language Crowdsourcing Instructions, Swaroop Mishra et al. arXiv reasoning

• FLEX: Unifying Evaluation for Few-Shot NLP, Jonathan Bragg et al. arXiv few-shot benchmark