1 code implementation • EMNLP (ACL) 2021 • Yash Kumar Lal, Reetu Singh, Harsh Trivedi, Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian
IrEne is an energy prediction system that accurately predicts the interpretable inference energy consumption of a wide range of Transformer-based NLP models.
1 code implementation • 20 Dec 2022 • Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal
While using the question to retrieve relevant text from an external knowledge source helps LLMs, we observe that this one-step retrieve-and-read approach is insufficient for multi-step QA.
no code implementations • 19 Oct 2022 • Alicia Parrish, Harsh Trivedi, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Amanpreet Singh Saimbhi, Samuel R. Bowman
These findings suggest that, in the case of answering reading comprehension questions, debate is not a helpful format.
1 code implementation • 5 Oct 2022 • Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, Ashish Sabharwal
On symbolic reasoning tasks, we can further decompose sub-tasks that are hard for LLMs into even simpler solvable sub-tasks.
1 code implementation • 25 May 2022 • Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal
We show how to use question decompositions to teach language models these broad reasoning skills in a robust fashion.
no code implementations • LNLS (ACL) 2022 • Alicia Parrish, Harsh Trivedi, Ethan Perez, Angelica Chen, Nikita Nangia, Jason Phang, Samuel R. Bowman
We use long contexts -- humans familiar with the context write convincing explanations for pre-selected correct and incorrect answers, and we test if those explanations allow humans who have not read the full context to more accurately determine the correct answer.
1 code implementation • EMNLP 2021 • Naoya Inoue, Harsh Trivedi, Steven Sinha, Niranjan Balasubramanian, Kentaro Inui
Instead, we advocate for an abstractive approach, where we propose to generate a question-focused, abstractive summary of input paragraphs and then feed it to an RC system.
2 code implementations • 2 Aug 2021 • Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal
Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts.
1 code implementation • ACL 2021 • Qingqing Cao, Yash Kumar Lal, Harsh Trivedi, Aruna Balasubramanian, Niranjan Balasubramanian
We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models.
1 code implementation • ACL 2021 • Nikita Nangia, Saku Sugawara, Harsh Trivedi, Alex Warstadt, Clara Vania, Samuel R. Bowman
However, we find that training crowdworkers, and then using an iterative process of collecting data, sending feedback, and qualifying workers based on expert judgments is an effective means of collecting challenging data.
1 code implementation • EMNLP 2020 • Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal
For a recent large-scale model (XLNet), we show that only 18 points out of its answer F1 score of 72 on HotpotQA are obtained through multifact reasoning, roughly the same as that of a simpler RNN baseline.
1 code implementation • ACL 2020 • Qingqing Cao, Harsh Trivedi, Aruna Balasubramanian, Niranjan Balasubramanian
It turns out that we can get by without input-wide self-attention at all layers, especially in the lower layers.
4 code implementations • NAACL 2019 • Harsh Trivedi, Heeyoung Kwon, Tushar Khot, Ashish Sabharwal, Niranjan Balasubramanian
We introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks.