no code implementations • WMT (EMNLP) 2021 • Shinhyeok Oh, Sion Jang, Hu Xu, Shounan An, Insoo Oh
As experimental results show, our APE system significantly improves the translations of provided MT results by -2. 848 and +3. 74 on the development dataset in terms of TER and BLEU, respectively.
no code implementations • 27 May 2024 • Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, Vasu Sharma, Hu Xu, Xiaoqing Ellen Tan, Megan Richards, Samuel Lavoie, Pietro Astolfi, Reyhane Askari Hemmat, Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, Vikas Chandra
Then, we present and discuss approaches to evaluate VLMs.
no code implementations • 26 Apr 2024 • Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Shang-Wen Li, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer
In recent times training Language Models (LMs) have relied on computationally heavy training over massive datasets which makes this training process extremely laborious.
1 code implementation • 24 Apr 2024 • Jiawei Ma, Po-Yao Huang, Saining Xie, Shang-Wen Li, Luke Zettlemoyer, Shih-Fu Chang, Wen-tau Yih, Hu Xu
The success of contrastive language-image pretraining (CLIP) relies on the supervision from the pairing between images and captions, which tends to be noisy in web-crawled data.
1 code implementation • 12 Mar 2024 • Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, Xian Li
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Ranked #30 on Question Answering on TriviaQA
2 code implementations • 28 Sep 2023 • Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, Christoph Feichtenhofer
We believe that the main ingredient to the success of CLIP is its data and not the model architecture or pre-training objective.
12 code implementations • 14 Apr 2023 • Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, Piotr Bojanowski
The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision.
Ranked #1 on Image Classification on CIFAR-10 (using extra training data)
no code implementations • ICCV 2023 • Chen Wei, Karttikeya Mangalam, Po-Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Cihang Xie, Alan Yuille, Christoph Feichtenhofer
There has been a longstanding belief that generation can facilitate a true understanding of visual data.
2 code implementations • 21 Jan 2023 • Zixuan Ke, Yijia Shao, Haowei Lin, Hu Xu, Lei Shu, Bing Liu
This paper shows that the existing methods are suboptimal and proposes a novel method to perform a more informed adaptation of the knowledge in the LM by (1) soft-masking the attention heads based on their importance to best preserve the general knowledge in the LM and (2) contrasting the representations of the general and the full (both general and domain knowledge) to learn an integrated representation with both general and domain-specific knowledge.
1 code implementation • ICCV 2023 • Hu Xu, Saining Xie, Po-Yao Huang, Licheng Yu, Russell Howes, Gargi Ghosh, Luke Zettlemoyer, Christoph Feichtenhofer
Large vision-language models are generally applicable to many downstream tasks, but come at an exorbitant training cost that only large institutions can afford.
1 code implementation • NeurIPS 2023 • Po-Yao Huang, Vasu Sharma, Hu Xu, Chaitanya Ryali, Haoqi Fan, Yanghao Li, Shang-Wen Li, Gargi Ghosh, Jitendra Malik, Christoph Feichtenhofer
We present Masked Audio-Video Learners (MAViL) to train audio-visual representations.
3 code implementations • 11 Oct 2022 • Zixuan Ke, Haowei Lin, Yijia Shao, Hu Xu, Lei Shu, Bing Liu
Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications.
Ranked #1 on Continual Pretraining on AG News
4 code implementations • 13 Jul 2022 • Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, Christoph Feichtenhofer
Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers.
Ranked #2 on Speaker Identification on VoxCeleb1 (using extra training data)
no code implementations • 4 Feb 2022 • Lei Shu, Hu Xu, Bing Liu, Jiahua Chen
Aspect-based sentiment analysis (ABSA) typically requires in-domain annotated data for supervised training/fine-tuning.
no code implementations • 19 Jan 2022 • Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer
We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens.
1 code implementation • NAACL 2021 • Zixuan Ke, Hu Xu, Bing Liu
This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks.
Ranked #3 on Continual Learning on ASC (19 tasks)
1 code implementation • NeurIPS 2021 • Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, Lei Shu
Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge.
Ranked #1 on Continual Learning on DSC (10 tasks)
1 code implementation • EMNLP 2021 • Zixuan Ke, Bing Liu, Hu Xu, Lei Shu
The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing.
2 code implementations • EMNLP 2021 • Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, Christoph Feichtenhofer
We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks.
Ranked #1 on Temporal Action Localization on CrossTask (using extra training data)
Action Segmentation Long Video Retrieval (Background Removed) +4
no code implementations • 14 Sep 2021 • Shinhyeok Oh, Sion Jang, Hu Xu, Shounan An, Insoo Oh
As experimental results show, our APE system significantly improves the translations of provided MT results by -2. 848 and +3. 74 on the development dataset in terms of TER and BLEU, respectively.
no code implementations • ICLR 2022 • Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, Luke Zettlemoyer
We introduce HTLM, a hyper-text language model trained on a large-scale web crawl.
Ranked #1 on Table-to-Text Generation on DART
1 code implementation • Findings (ACL) 2021 • Hu Xu, Gargi Ghosh, Po-Yao Huang, Prahal Arora, Masoumeh Aminzadeh, Christoph Feichtenhofer, Florian Metze, Luke Zettlemoyer
We present a simplified, task-agnostic multi-modal pre-training approach that can accept either video or text input, or both for a variety of end tasks.
Ranked #2 on Temporal Action Localization on CrossTask (using extra training data)
no code implementations • 27 Mar 2021 • Kun-Peng Ning, Hu Xu, Kun Zhu, Sheng-Jun Huang
Imitation learning is a primary approach to improve the efficiency of reinforcement learning by exploiting the expert demonstrations.
no code implementations • ICCV 2021 • Yuwei Cheng, Hu Xu, Yimin Liu
In our work, we focus on a relatively unexplored task for USVs in inland waters: small object detection on water surfaces, which is of vital importance for safe autonomous navigation and USVs' certain missions such as floating waste cleaning.
2 code implementations • COLING 2020 • Hu Xu, Lei Shu, Philip S. Yu, Bing Liu
Most features in the representation of an aspect are dedicated to the fine-grained semantics of the domain (or product category) and the aspect itself, instead of carrying summarized opinions from its context.
Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA) +2
1 code implementation • Findings (EMNLP) 2021 • Zhiyu Chen, Honglei Liu, Hu Xu, Seungwhan Moon, Hao Zhou, Bing Liu
As there is no clean mapping for a user's free form utterance to an ontology, we first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Lei Shu, Alexandros Papangelis, Yi-Chia Wang, Gokhan Tur, Hu Xu, Zhaleh Feizollahi, Bing Liu, Piero Molino
This work introduces Focused-Variation Network (FVN), a novel model to control language generation.
no code implementations • COLING 2020 • Hu Xu, Seungwhan Moon, Honglei Liu, Pararth Shah, Bing Liu, Philip S. Yu
We study a conversational recommendation model which dynamically manages users' past (offline) preferences and current (online) requests through a structured and cumulative user memory knowledge graph, to allow for natural interactions and accurate recommendations.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
This paper focuses on learning domain-oriented language models driven by end tasks, which aims to combine the worlds of both general-purpose language models (such as ELMo and BERT) and domain-specific language understanding.
Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA) +1
1 code implementation • 4 Nov 2019 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Aspect-based sentiment classification (ASC) is an important task in fine-grained sentiment analysis.~Deep supervised ASC approaches typically model this task as a pair-wise classification task that takes an aspect and a sentence containing the aspect and outputs the polarity of the aspect in that sentence.
1 code implementation • IJCNLP 2019 • Lei Shu, Hu Xu, Bing Liu, Piero Molino
Dialogue management (DM) plays a key role in the quality of the interaction with the user in a task-oriented dialogue system.
1 code implementation • WS 2019 • Lei Shu, Piero Molino, Mahdi Namazifar, Hu Xu, Bing Liu, Huaixiu Zheng, Gokhan Tur
It is based on a simple and practical yet very effective sequence-to-sequence approach, where language understanding and state tracking tasks are modeled jointly with a structured copy-augmented sequential decoder and a multi-label decoder for each slot.
no code implementations • 15 May 2019 • Lei Shu, Hu Xu, Bing Liu
The modified CNN has two types of control modules.
1 code implementation • NAACL 2019 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Since ReviewRC has limited training examples for RRC (and also for aspect-based sentiment analysis), we then explore a novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC.
1 code implementation • 3 Feb 2019 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Inspired by conversational reading comprehension (CRC), this paper studies a novel task of leveraging reviews as a source to build an agent that can answer multi-turn questions from potential consumers of online businesses.
1 code implementation • 17 Sep 2018 • Hu Xu, Bing Liu, Lei Shu, P. Yu
Classic supervised learning makes the closed-world assumption, meaning that classes seen in testing must have been seen in training.
1 code implementation • 25 May 2018 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Learning high-quality domain word embeddings is important for achieving good performance in many NLP tasks.
2 code implementations • ACL 2018 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Unlike other highly sophisticated supervised deep learning models, this paper proposes a novel and yet simple CNN model employing two types of pre-trained embeddings for aspect extraction: general-purpose embeddings and domain-specific embeddings.
1 code implementation • ICLR 2018 • Lei Shu, Hu Xu, Bing Liu
It is reasonable to assume that this knowledge can be transferred to the rejected examples and used to discover the hidden unseen classes in them.
no code implementations • ICLR 2018 • Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
We observe that domains are not isolated and a small domain corpus can leverage the learned knowledge from many past domains to augment that corpus in order to generate high-quality embeddings.
no code implementations • 6 Dec 2017 • Hu Xu, Sihong Xie, Lei Shu, Philip S. Yu
Product compatibility and their functionality are of utmost importance to customers when they purchase products, and to sellers and manufacturers when they sell products.
no code implementations • 6 Dec 2017 • Hu Xu, Sihong Xie, Lei Shu, Philip S. Yu
Functionality is of utmost importance to customers when they purchase products.
no code implementations • EMNLP 2017 • Lei Shu, Hu Xu, Bing Liu
As learning is used increasingly in dynamic open environments where some new/test documents may not belong to any of the training classes, identifying these novel documents during classification presents an important problem.
no code implementations • 29 May 2017 • Hu Xu, Lei Shu, Philip S. Yu
Extracting opinion targets is an important task in sentiment analysis on product reviews and complementary entities (products) are one important type of opinion targets that may work together with the reviewed product.
no code implementations • ACL 2017 • Lei Shu, Hu Xu, Bing Liu
This paper makes a focused contribution to supervised aspect extraction.
no code implementations • 23 Dec 2016 • Lei Shu, Bing Liu, Hu Xu, Annice Kim
When "screen" appears in a review of a new domain (or product), it is likely to be an aspect too.
no code implementations • 14 Dec 2016 • Hu Xu, Lei Shu, Jingyuan Zhang, Philip S. Yu
In this paper, we address the problem of extracting compatible and incompatible products from yes/no questions in PCQA.
no code implementations • 4 Dec 2016 • Hu Xu, Sihong Xie, Lei Shu, Philip S. Yu
One important product feature is the complementary entity (products) that may potentially work together with the reviewed product.