no code implementations • COLING 2022 • Yang Sun, Liangqing Wu, Shuangyong Song, Xiaoguang Yu, Xiaodong He, Guohong Fu
In this work, we investigate the problem of satisfaction states tracking and its effects on CSP in E-commerce service chatbots.
1 code implementation • EMNLP 2021 • Haoran Li, Song Xu, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, BoWen Zhou
It thereby takes advantage of prior copying distributions and, at each time step, explicitly encourages the model to copy the input word that is relevant to the previously copied one.
Ranked #11 on Abstractive Text Summarization on CNN / Daily Mail (using extra training data)
no code implementations • COLING 2022 • Ruixue Liu, Shaozu Yuan, Aijun Dai, Lei Shen, Tiangang Zhu, Meng Chen, Xiaodong He
Since there is no large number of public Chinese tables, we also collect a large-scale, multi-domain tabular corpus to facilitate future Chinese table pre-training, which includes one million tables and related natural language text with auxiliary supervised interaction signals.
no code implementations • LREC 2022 • Meihuizi Jia, Ruixue Liu, Peiying Wang, Yang song, Zexi Xi, Haobin Li, Xin Shen, Meng Chen, Jinhui Pang, Xiaodong He
There has been a growing interest in developing conversational recommendation system (CRS), which provides valuable recommendations to users through conversations.
1 code implementation • NAACL 2022 • Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
1 code implementation • NAACL 2022 • Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang, Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao
To inherit the advantages of these two types of methods, we propose OPERA, an operation-pivoted discrete reasoning framework, where lightweight symbolic operations (compared with logical forms) as neural modules are utilized to facilitate the reasoning ability and interpretability.
no code implementations • 24 Dec 2023 • Guanqun Bi, Lei Shen, Yuqiang Xie, Yanan Cao, Tiangang Zhu, Xiaodong He
The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts.
1 code implementation • 28 Nov 2023 • Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lusong Li, Li Shen, Xiaodong He, Jing Jiang, Yuhui Shi
While large language models (LLMs) excel in a simulated world of texts, they struggle to interact with the more realistic world without perceptions of other modalities such as visual or audio signals.
no code implementations • 5 Sep 2023 • Peiying Wang, Sunlu Zeng, Junqing Chen, Lu Fan, Meng Chen, Youzheng Wu, Xiaodong He
Finally, we devise a novel label-guided attentive fusion module to fuse the label-aware text and speech representations for emotion classification.
no code implementations • 16 Jun 2023 • Yu Lu, Junwei Bao, Zichen Ma, Xiaoguang Han, Youzheng Wu, Shuguang Cui, Xiaodong He
High-quality data is essential for conversational recommendation systems and serves as the cornerstone of the network architecture development and training strategy design.
no code implementations • 5 Jun 2023 • Li Fu, Siqi Li, Qingtao Li, Fangzhu Li, Liping Deng, Lu Fan, Meng Chen, Youzheng Wu, Xiaodong He
Self-Supervised Learning (SSL) Automatic Speech Recognition (ASR) models have shown great promise over Supervised Learning (SL) ones in low-resource settings.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 2 Jun 2023 • Guanqun Bi, Lei Shen, Yanan Cao, Meng Chen, Yuqiang Xie, Zheng Lin, Xiaodong He
Empathy is a crucial factor in open-domain conversations, which naturally shows one's caring and understanding to others.
no code implementations • 8 May 2023 • Shaozu Yuan, Aijun Dai, Zhiling Yan, Ruixue Liu, Meng Chen, Baoyang Chen, Zhijie Qiu, Xiaodong He
In this paper, we present a novel system (denoted as Polaca) to generate poetic Chinese landscape painting with calligraphy.
no code implementations • 23 Feb 2023 • Xiaodong He, Zhongkui Li, Xiangke Wang, Zhiyong Geng
Secondly, given feasible formations, we design a formation controller by introducing a virtual leader and employing the compensation of rotation, followed by proving the stability of the closed-loop system.
no code implementations • 27 Nov 2022 • Meihuizi Jia, Lei Shen, Xin Shen, Lejian Liao, Meng Chen, Xiaodong He, Zhendong Chen, Jiaqi Li
Multimodal named entity recognition (MNER) is a critical step in information extraction, which aims to detect entity spans and classify them to corresponding entity types given a sentence-image pair.
1 code implementation • 27 Nov 2022 • Huaishao Luo, Junwei Bao, Youzheng Wu, Xiaodong He, Tianrui Li
The pre-trained model can capture enriched visual concepts for images by learning from a large scale of text-image data.
Ranked #1 on Semantic Segmentation on PASCAL VOC
no code implementations • 10 Nov 2022 • Haoning Zhang, Junwei Bao, Haipeng Sun, Youzheng Wu, Wenye Li, Shuguang Cui, Xiaodong He
Then, the noised previous state is used as the input to learn to predict the current state, improving the model's ability to update and correct slot values.
no code implementations • 26 Oct 2022 • Li Fu, Siqi Li, Qingtao Li, Liping Deng, Fangzhu Li, Lu Fan, Meng Chen, Xiaodong He
In this paper, we propose a Unified pre-training Framework for Online and Offline (UFO2) Automatic Speech Recognition (ASR), which 1) simplifies the two separate training workflows for online and offline modes into one process, and 2) improves the Word Error Rate (WER) performance with limited utterance annotating.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 22 Oct 2022 • Junwei Bao, Yifan Wang, Jiangyong Ying, Yeyun Gong, Jing Zhao, Youzheng Wu, Xiaodong He
Conventional autoregressive left-to-right (L2R) sequence generation faces two issues during decoding: limited to unidirectional target sequence modeling, and constrained on strong local dependencies.
1 code implementation • 19 Oct 2022 • Yingyao Wang, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He, Tiejun Zhao
To preserve the advantage and eliminate the disadvantage of different granularity evidence, we propose MuGER$^2$, a Multi-Granularity Evidence Retrieval and Reasoning approach.
1 code implementation • 17 Oct 2022 • Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He
Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response.
1 code implementation • 15 Oct 2022 • Yongwei Zhou, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He, Tiejun Zhao
Question answering requiring discrete reasoning, e. g., arithmetic computing, comparison, and counting, over knowledge is a challenging task.
1 code implementation • 26 Aug 2022 • Guanming Xiong, Junwei Bao, Wen Zhao, Youzheng Wu, Xiaodong He
This study investigates the task of knowledge-based question generation (KBQG).
1 code implementation • 1 Aug 2022 • Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, Zhiting Hu
This paper proposes a new efficient approach for composable text operations in the compact latent space of text.
Ranked #2 on Unsupervised Text Style Transfer on Yelp
no code implementations • 10 May 2022 • Xiaodong He, Yinan Wang, Juan Li
This paper introduces a novel neural network - flow completion network (FCN) - to infer the fluid dynamics, includ-ing the flow field and the force acting on the body, from the incomplete data based on Graph Convolution AttentionNetwork.
1 code implementation • Findings (NAACL) 2022 • Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He
To enhance the denoising capability of the model to reduce the impact of error propagation, denoising reconstruction is used to reconstruct the corrupted dialog state and response.
1 code implementation • NAACL 2022 • Yifan Wang, Jing Zhao, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He
Dialogue state tracking (DST) aims to predict the current dialogue state given the dialogue history.
no code implementations • 29 Apr 2022 • Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang, Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao
To inherit the advantages of these two types of methods, we propose OPERA, an operation-pivoted discrete reasoning framework, where lightweight symbolic operations (compared with logical forms) as neural modules are utilized to facilitate the reasoning ability and interpretability.
no code implementations • 27 Apr 2022 • Lin Yang, Shuai Guo, Chengyu Houc, Jiacheng Lia, Liping Shi, Chenchen Liao, Rongchun Shi, Xiaoliang Ma, Bing Zheng, Yi Fang, Lin Ye, Xiaodong He
The low-entropy level of hydration shells at the binding site of a spike protein is found to be an important indicator of the contagiousness of the coronavirus.
no code implementations • NAACL 2022 • Zhenyu Zhang, Yuming Zhao, Meng Chen, Xiaodong He
Motivated by this, we propose a novel label anchored contrastive learning approach (denoted as LaCon) for language understanding.
no code implementations • 22 Apr 2022 • Shaozu Yuan, Ruixue Liu, Meng Chen, Baoyang Chen, Zhijie Qiu, Xiaodong He
There is rare research on brush handwriting font generation, which involves holistic structure changes and complex strokes transfer.
no code implementations • 18 Apr 2022 • Jiudong Yang, Peiying Wang, Yi Zhu, Mingchao Feng, Meng Chen, Xiaodong He
Turn-taking, aiming to decide when the next speaker can start talking, is an essential component in building human-robot spoken dialogue systems.
no code implementations • 26 Mar 2022 • Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han, Zhenghao Liu, Ning Ding, Yongming Rao, Yizhao Gao, Liang Zhang, Ming Ding, Cong Fang, Yisen Wang, Mingsheng Long, Jing Zhang, Yinpeng Dong, Tianyu Pang, Peng Cui, Lingxiao Huang, Zheng Liang, HuaWei Shen, HUI ZHANG, Quanshi Zhang, Qingxiu Dong, Zhixing Tan, Mingxuan Wang, Shuo Wang, Long Zhou, Haoran Li, Junwei Bao, Yingwei Pan, Weinan Zhang, Zhou Yu, Rui Yan, Chence Shi, Minghao Xu, Zuobai Zhang, Guoqiang Wang, Xiang Pan, Mengjie Li, Xiaoyu Chu, Zijun Yao, Fangwei Zhu, Shulin Cao, Weicheng Xue, Zixuan Ma, Zhengyan Zhang, Shengding Hu, Yujia Qin, Chaojun Xiao, Zheni Zeng, Ganqu Cui, Weize Chen, Weilin Zhao, Yuan YAO, Peng Li, Wenzhao Zheng, Wenliang Zhao, Ziyi Wang, Borui Zhang, Nanyi Fei, Anwen Hu, Zenan Ling, Haoyang Li, Boxi Cao, Xianpei Han, Weidong Zhan, Baobao Chang, Hao Sun, Jiawen Deng, Chujie Zheng, Juanzi Li, Lei Hou, Xigang Cao, Jidong Zhai, Zhiyuan Liu, Maosong Sun, Jiwen Lu, Zhiwu Lu, Qin Jin, Ruihua Song, Ji-Rong Wen, Zhouchen Lin, LiWei Wang, Hang Su, Jun Zhu, Zhifang Sui, Jiajun Zhang, Yang Liu, Xiaodong He, Minlie Huang, Jian Tang, Jie Tang
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
no code implementations • 22 Mar 2022 • Zexun Wang, Yuquan Le, Yi Zhu, Yuming Zhao, Mingchao Feng, Meng Chen, Xiaodong He
Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
1 code implementation • ACL 2022 • Jing Zhao, Yifan Wang, Junwei Bao, Youzheng Wu, Xiaodong He
To confront this, we propose FCA, a fine- and coarse-granularity hybrid self-attention that reduces the computation cost through progressively shortening the computational sequence length in self-attention.
no code implementations • 22 Feb 2022 • Lin Yang, Shuai Guo, Chengyu Hou, Chencheng Liao, Jiacheng Li, Liping Shi, Xiaoliang Ma, Shenda Jiang, Bing Zheng, Yi Fang, Lin Ye, Xiaodong He
According to an analysis of determined protein complex structures, shape matching between the largest low-entropy hydration shell region of a protein and that of its partner at the binding sites is revealed as a regular pattern.
no code implementations • 18 Jan 2022 • Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo
In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation.
no code implementations • 27 Dec 2021 • Yuan YAO, Qingxiu Dong, Jian Guan, Boxi Cao, Zhengyan Zhang, Chaojun Xiao, Xiaozhi Wang, Fanchao Qi, Junwei Bao, Jinran Nie, Zheni Zeng, Yuxian Gu, Kun Zhou, Xuancheng Huang, Wenhao Li, Shuhuai Ren, Jinliang Lu, Chengqiang Xu, Huadong Wang, Guoyang Zeng, Zile Zhou, Jiajun Zhang, Juanzi Li, Minlie Huang, Rui Yan, Xiaodong He, Xiaojun Wan, Xin Zhao, Xu sun, Yang Liu, Zhiyuan Liu, Xianpei Han, Erhong Yang, Zhifang Sui, Maosong Sun
We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic.
no code implementations • 26 Oct 2021 • Tong Shen, Jiawei Zuo, Fan Shi, Jin Zhang, Liqin Jiang, Meng Chen, Zhengchen Zhang, Wei zhang, Xiaodong He, Tao Mei
We demonstrate ViDA-MAN, a digital-human agent for multi-modal interaction, which offers realtime audio-visual responses to instant speech inquiries.
no code implementations • 8 Oct 2021 • Li Fu, Xiaoxiao Li, Runyu Wang, Lu Fan, Zhengchen Zhang, Meng Chen, Youzheng Wu, Xiaodong He
End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervision.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +3
no code implementations • 27 Sep 2021 • Nan Zhao, Haoran Li, Youzheng Wu, Xiaodong He, BoWen Zhou
We present the solutions of top-5 teams participating in the JDDC multimodal dialogue challenge based on this dataset, which provides valuable insights for further researches on the multimodal dialogue task.
1 code implementation • Findings (EMNLP) 2021 • Jing Zhao, Junwei Bao, Yifan Wang, Yongwei Zhou, Youzheng Wu, Xiaodong He, BoWen Zhou
To address this problem, we propose RoR, a read-over-read method, which expands the reading field from chunk to document.
1 code implementation • 18 Aug 2021 • Yongwei Zhou, Junwei Bao, Haipeng Sun, Jiahui Liang, Youzheng Wu, Xiaodong He, BoWen Zhou, Tiejun Zhao
Reasoning machine reading comprehension (R-MRC) aims to answer complex questions that require discrete reasoning based on text.
1 code implementation • 18 Aug 2021 • Jiahui Liang, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, BoWen Zhou
To address the problem, we propose CUSTOM, aspect-oriented product summarization for e-commerce, which generates diverse and controllable summaries towards different product aspects.
1 code implementation • 29 Jun 2021 • Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
no code implementations • 9 Jun 2021 • Zichuan Lin, Jing Huang, BoWen Zhou, Xiaodong He, Tengyu Ma
Recent work (Takanobu et al., 2020) proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e. g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation.
1 code implementation • Findings (ACL) 2021 • Yu Lu, Junwei Bao, Yan Song, Zichen Ma, Shuguang Cui, Youzheng Wu, Xiaodong He
Existing conversational recommendation (CR) systems usually suffer from insufficient item information when conducted on short dialogue history and unfamiliar items.
no code implementations • 13 May 2021 • Peng Qi, Jing Huang, Youzheng Wu, Xiaodong He, BoWen Zhou
Conversational artificial intelligence (ConvAI) systems have attracted much academic and commercial attention recently, making significant progress on both fronts.
1 code implementation • NAACL 2021 • Jing Zhao, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, BoWen Zhou
Keyphrases, that concisely summarize the high-level topics discussed in a document, can be categorized into present keyphrase which explicitly appears in the source text, and absent keyphrase which does not match any contiguous subsequence but is highly semantically related to the source.
1 code implementation • Findings (EMNLP) 2021 • Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, BoWen Zhou
K-PLUG achieves new state-of-the-art results on a suite of domain-specific NLP tasks, including product knowledge base completion, abstractive product summarization, and multi-turn dialogue, significantly outperforms baselines across the board, which demonstrates that the proposed method effectively learns a diverse set of domain-specific knowledge for both language understanding and generation tasks.
no code implementations • NAACL 2021 • Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, BoWen Zhou
Recent work on aspect-level sentiment classification has demonstrated the efficacy of incorporating syntactic structures such as dependency trees with graph neural networks(GNN), but these approaches are usually vulnerable to parsing errors.
no code implementations • 28 Feb 2021 • Jiacheng Li, Chengyu Hou, Menghao Wang, Chencheng Liao, Shuai Guo, Liping Shi, Xiaoliang Ma, Hongchi Zhang, Shenda Jiang, Bing Zheng, Lin Ye, Lin Yang, Xiaodong He
Preliminary epidemiologic, phylogenetic and clinical findings suggest that several novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants have increased transmissibility and decreased efficacy of several existing vaccines.
no code implementations • 9 Feb 2021 • Hang Liu, Meng Chen, Youzheng Wu, Xiaodong He, BoWen Zhou
Conversational Query Rewriting (CQR) aims to simplify the multi-turn dialogue modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance.
1 code implementation • 1 Jan 2021 • Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, BoWen Zhou
K-PLUG achieves new state-of-the-art results on a suite of domain-specific NLP tasks, including product knowledge base completion, abstractive product summarization, and multi-turn dialogue, significantly outperforms baselines across the board, which demonstrates that the proposed method effectively learns a diverse set of domain-specific knowledge for both language understanding and generation tasks.
1 code implementation • NeurIPS 2020 • Xu Liu, Chengtao Li, Jian Wang, Jingbo Wang, Boxin Shi, Xiaodong He
In this work, we extended the contextual encoding layer that was originally designed for 2D tasks to 3D Point Cloud scenarios.
no code implementations • COLING 2020 • Haoran Li, Junnan Zhu, Jiajun Zhang, Xiaodong He, Chengqing Zong
Thus, we propose a multimodal selective gate network that considers reciprocal relationships between textual and multi-level visual features, including global image descriptor, activation grids, and object proposals, to select highlights of the event when encoding the source sentence.
1 code implementation • COLING 2020 • Peng Yuan, Haoran Li, Song Xu, Youzheng Wu, Xiaodong He, BoWen Zhou
In this work, we present a model to generate e-commerce product summaries.
no code implementations • 6 Nov 2020 • Guanghui Xu, Wei Song, Zhengchen Zhang, Chao Zhang, Xiaodong He, BoWen Zhou
Despite prosody is related to the linguistic information up to the discourse structure, most text-to-speech (TTS) systems only take into account that within each sentence, which makes it challenging when converting a paragraph of texts into natural and expressive speech.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Ruosong Yang, Jiannong Cao, Zhiyuan Wen, Youzheng Wu, Xiaodong He
However, to solve the AES task, previous works utilize shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss, respectively.
no code implementations • COLING 2020 • Yingyao Wang, Junwei Bao, Guangyi Liu, Youzheng Wu, Xiaodong He, BoWen Zhou, Tiejun Zhao
This paper aims to enhance the few-shot relation classification especially for sentences that jointly describe multiple relations.
no code implementations • 16 Sep 2020 • Jiacheng Li, Xiaoliang Ma, Hongchi Zhang, Chengyu Hou, Liping Shi, Shuai Guo, Chenchen Liao, Bing Zheng, Lin Ye, Lin Yang, Xiaodong He
Exploring the protein-folding problem has been a long-standing challenge in molecular biology.
2 code implementations • EMNLP 2020 • Tiangang Zhu, Yue Wang, Haoran Li, Youzheng Wu, Xiaodong He, Bo-Wen Zhou
We annotate a multimodal product attribute value dataset that contains 87, 194 instances, and the experimental results on this dataset demonstrate that explicitly modeling the relationship between attributes and values facilitates our method to establish the correspondence between them, and selectively utilizing visual product information is necessary for the task.
no code implementations • 27 Aug 2020 • Jiacheng Li, Xiaoliang Ma, Shuai Guo, Chengyu Hou, Liping Shi, Hongchi Zhang, Bing Zheng, Chencheng Liao, Lin Yang, Lin Ye, Xiaodong He
The hydrophobic interaction between the SARS-CoV-2 S and ACE2 protein is found to be significantly greater than that between SARS-CoV S and ACE2.
no code implementations • 28 Jul 2020 • Wei Xue, Gang Quan, Chao Zhang, Guohong Ding, Xiaodong He, BoWen Zhou
Statistical signal processing based speech enhancement methods adopt expert knowledge to design the statistical models and linear filters, which is complementary to the deep neural network (DNN) based methods which are data-driven.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • ACL 2020 • Song Xu, Haoran Li, Peng Yuan, Youzheng Wu, Xiaodong He, Bo-Wen Zhou
Copy module has been widely equipped in the recent abstractive summarization models, which facilitates the decoder to extract words from the source into the summary.
no code implementations • 11 May 2020 • Li Fu, Xiaoxiao Li, Libo Zi, Zhengchen Zhang, Youzheng Wu, Xiaodong He, BoWen Zhou
In this paper, we propose an incremental learning method for end-to-end Automatic Speech Recognition (ASR) which enables an ASR system to perform well on new tasks while maintaining the performance on its originally learned ones.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +3
no code implementations • 13 Apr 2020 • Tae Jin Park, Kyu J. Han, Jing Huang, Xiaodong He, Bo-Wen Zhou, Panayiotis Georgiou, Shrikanth Narayanan
This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
no code implementations • 4 Apr 2020 • Ming Tu, Jing Huang, Xiaodong He, Bo-Wen Zhou
We validate the proposed GSN on two NLP tasks: interpretable multi-hop reading comprehension on HotpotQA and graph based fact verification on FEVER.
no code implementations • 10 Dec 2019 • Yi Zhou, Boyang Wang, Xiaodong He, Shanshan Cui, Ling Shao
In this paper, we propose a diabetic retinopathy generative adversarial network (DR-GAN) to synthesize high-resolution fundus images which can be manipulated with arbitrary grading and lesion information.
no code implementations • LREC 2020 • Meng Chen, Ruixue Liu, Lei Shen, Shaozu Yuan, Jingyan Zhou, Youzheng Wu, Xiaodong He, Bo-Wen Zhou
Human conversations are complicated and building a human-like dialogue agent is an extremely challenging task.
no code implementations • 10 Nov 2019 • Chao Zhang, Zichao Yang, Xiaodong He, Li Deng
This review provides a comprehensive analysis of recent works on multimodal deep learning from three perspectives: learning multimodal representations, fusing multimodal signals at various levels, and multimodal applications.
no code implementations • ACL 2020 • Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bo-Wen Zhou
Translational distance-based knowledge graph embedding has shown progressive improvements on the link prediction task, from TransE to the latest state-of-the-art RotatE.
Ranked #18 on Link Prediction on FB15k-237
no code implementations • CONLL 2019 • Kevin Huang, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou
We test the relation module on the SQuAD 2. 0 dataset using both the BiDAF and BERT models as baseline readers.
1 code implementation • 1 Nov 2019 • Ming Tu, Kevin Huang, Guangtao Wang, Jing Huang, Xiaodong He, Bo-Wen Zhou
Interpretable multi-hop reading comprehension (RC) over multiple documents is a challenging problem because it demands reasoning over multiple information sources and explaining the answer prediction by providing supporting evidences.
no code implementations • NAACL (TextGraphs) 2021 • Xiaochen Hou, Jing Huang, Guangtao Wang, Xiaodong He, BoWen Zhou
Aspect-level sentiment classification aims to identify the sentiment polarity towards a specific aspect term in a sentence.
no code implementations • 23 Oct 2019 • Kevin Huang, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou
In this paper, we aim to improve a MRC model's ability to determine whether a question has an answer in a given context (e. g. the recently proposed SQuAD 2. 0 task).
1 code implementation • 29 Aug 2019 • Shuaichen Chang, PengFei Liu, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou
Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task.
no code implementations • 12 Jun 2019 • Ming Tu, Jing Huang, Xiaodong He, Bo-Wen Zhou
In this paper, we propose a new end-to-end graph neural network (GNN) based algorithm for MIL: we treat each bag as a graph and use GNN to learn the bag embedding, in order to explore the useful structural information among instances in bags.
no code implementations • ACL 2019 • Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, Bo-Wen Zhou
We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph.
1 code implementation • NeurIPS 2019 • Fenglin Liu, Yuanxin Liu, Xuancheng Ren, Xiaodong He, Xu sun
In vision-and-language grounding problems, fine-grained representations of the image are considered to be of paramount importance.
no code implementations • 9 May 2019 • Ruixue Liu, Baoyang Chen, Meng Chen, Youzheng Wu, Zhijie Qiu, Xiaodong He
We present a novel real-time, collaborative, and interactive AI painting system, Mappa Mundi, for artistic Mind Map creation.
1 code implementation • IJCAI 2019 2019 • Pengcheng Yang, Fuli Luo, Peng Chen, Lei LI, Zhiyi Yin, Xiaodong He, Xu sun
The visual storytelling (VST) task aims at generating a reasonable and coherent paragraph-level story with the image stream as input.
Ranked #21 on Visual Storytelling on VIST
no code implementations • 4 Mar 2019 • Ruixue Liu, Baoyang Chen, XIAOYU GUO, Yan Dai, Meng Chen, Zhijie Qiu, Xiaodong He
Imagination is one of the most important factors which makes an artistic painting unique and impressive.
1 code implementation • CVPR 2019 • Wenbo Li, Pengchuan Zhang, Lei Zhang, Qiuyuan Huang, Xiaodong He, Siwei Lyu, Jianfeng Gao
In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow object-centered text-to-image synthesis for complex scenes.
no code implementations • 21 Feb 2019 • Yun Tang, Guohong Ding, Jing Huang, Xiaodong He, Bo-Wen Zhou
This paper aims to improve the widely used deep speaker embedding x-vector model.
1 code implementation • 11 Nov 2018 • Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, Bo-Wen Zhou
The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure.
Ranked #28 on Link Prediction on FB15k-237
no code implementations • EMNLP 2018 • Dipendra Misra, Ming-Wei Chang, Xiaodong He, Wen-tau Yih
Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e. g., answers), search for good candidate semantic parses, and (2) choose the best model update algorithm.
no code implementations • ACL 2018 • William Yang Wang, Jiwei Li, Xiaodong He
Many Natural Language Processing (NLP) tasks (including generation, language grounding, reasoning, information extraction, coreference resolution, and dialog) can be formulated as deep reinforcement learning (DRL) problems.
no code implementations • 21 May 2018 • Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Wu, Jian-Feng Wang, Xiaodong He
We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task.
Ranked #24 on Visual Storytelling on VIST
no code implementations • NAACL 2018 • Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi
In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text.
1 code implementation • 3 Apr 2018 • Dianqi Li, Qiuyuan Huang, Xiaodong He, Lei Zhang, Ming-Ting Sun
By contrasting with human-written captions and image-mismatched captions, the caption generator effectively exploits the inherent characteristics of human languages, and generates more discriminative captions.
no code implementations • NAACL 2018 • Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi
We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization.
Ranked #31 on Abstractive Text Summarization on CNN / Daily Mail (using extra training data)
6 code implementations • ECCV 2018 • Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He
Prior work either simply aggregates the similarity of all possible pairs of regions and words without attending differentially to more and less important words or regions, or uses a multi-step attentional process to capture limited number of semantic alignments which is less interpretable.
Ranked #4 on Image Retrieval on PhotoChat
1 code implementation • NAACL 2018 • Po-Sen Huang, Chenglong Wang, Rishabh Singh, Wen-tau Yih, Xiaodong He
In conventional supervised training, a model is trained to fit all the training examples.
Ranked #7 on Code Generation on WikiSQL
no code implementations • 20 Feb 2018 • Qiuyuan Huang, Li Deng, Dapeng Wu, Chang Liu, Xiaodong He
This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models.
no code implementations • 16 Feb 2018 • Rasool Fakoor, Xiaodong He, Ivan Tashev, Shuayb Zarar
For a speech-enhancement algorithm, it is highly desirable to simultaneously improve perceptual quality and recognition rate.
no code implementations • 6 Jan 2018 • Heung-Yeung Shum, Xiaodong He, Di Li
Conversational systems have come a long way since their inception in the 1960s.
no code implementations • 29 Nov 2017 • Rasool Fakoor, Xiaodong He, Ivan Tashev, Shuayb Zarar
Today, the optimal performance of existing noise-suppression algorithms, both data-driven and those based on classic statistical methods, is range bound to specific levels of instantaneous input signal-to-noise ratios.
19 code implementations • CVPR 2018 • Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He
In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation.
Ranked #1 on Text-to-Image Generation on MS-COCO
3 code implementations • CVPR 2018 • Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang
We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets.
Ranked #2 on Image Classification on Food-101N (using extra training data)
no code implementations • ICLR 2018 • Pengchuan Zhang, Qiang Liu, Dengyong Zhou, Tao Xu, Xiaodong He
When evaluated with neural distance, our bounds show that generalization is guaranteed as long as the discriminator set is small enough, regardless of the size of the generator or hypothesis set.
no code implementations • 29 Oct 2017 • Qiuyuan Huang, Paul Smolensky, Xiaodong He, Li Deng, Dapeng Wu
To address this, this paper promotes image/visual captioning based CAPTCHAs, which is robust against machine-learning-based attacks.
2 code implementations • NAACL 2018 • Qiuyuan Huang, Paul Smolensky, Xiaodong He, Li Deng, Dapeng Wu
We present a new approach to the design of deep networks for natural language processing (NLP), based on the general technique of Tensor Product Representations (TPRs) for encoding and processing symbol structures in distributed neural networks.
no code implementations • 22 Aug 2017 • Zheng Tang, Gaoang Wang, Tao Liu, Young-Gun Lee, Adwin Jahn, Xu Liu, Xiaodong He, Jenq-Neng Hwang
In this challenge, we propose a model-based vehicle localization method, which builds a kernel at each patch of the 3D deformable vehicle model and associates them with constraints in 3D space.
10 code implementations • CVPR 2018 • Damien Teney, Peter Anderson, Xiaodong He, Anton Van Den Hengel
This paper presents a state-of-the-art model for visual question answering (VQA), which won the first place in the 2017 VQA Challenge.
Ranked #30 on Visual Question Answering (VQA) on VQA v2 test-std
65 code implementations • CVPR 2018 • Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang
Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained analysis and even multiple steps of reasoning.
Ranked #29 on Visual Question Answering (VQA) on VQA v2 test-std
no code implementations • CVPR 2017 • Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, Li Deng
We propose a novel framework named StyleNet to address the task of generating attractive captions for images and videos with different styles.
2 code implementations • EMNLP 2017 • David Golub, Po-Sen Huang, Xiaodong He, Li Deng
We develop a technique for transfer learning in machine comprehension (MC) using a novel two-stage synthesis network (SynNet).
1 code implementation • NeurIPS 2017 • Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, Ming-Ting Sun
Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group.
Ranked #1 on Text Generation on Chinese Poems
no code implementations • 23 May 2017 • Hamid Palangi, Paul Smolensky, Xiaodong He, Li Deng
In our application of TPRN, internal representations learned by end-to-end optimization in a deep neural network performing a textual question-answering (QA) task can be interpreted using basic concepts from linguistic theory.
no code implementations • 20 Apr 2017 • Ji He, Mari Ostendorf, Xiaodong He
This paper addresses the problem of predicting popularity of comments in an online discussion forum using reinforcement learning, particularly addressing two challenges that arise from having natural language state and action spaces.
2 code implementations • 8 Feb 2017 • Zhe Gan, P. D. Singh, Ameet Joshi, Xiaodong He, Jianshu Chen, Jianfeng Gao, Li Deng
Connecting different text attributes associated with the same entity (conflation) is important in business data analytics since it could help merge two different tables in a database to provide a more comprehensive profile of an entity.
1 code implementation • CVPR 2017 • Zhaowei Cai, Xiaodong He, Jian Sun, Nuno Vasconcelos
The problem of quantizing the activations of a deep neural network is considered.
1 code implementation • CVPR 2017 • Zhe Gan, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, Li Deng
The degree to which each member of the ensemble is used to generate an image caption is tied to the image-dependent probability of the corresponding tag.
no code implementations • EMNLP 2017 • Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin
We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes.
1 code implementation • EMNLP 2016 • Hao Cheng, Hao Fang, Xiaodong He, Jianfeng Gao, Li Deng
We develop a novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions.
Ranked #4 on Chinese Dependency Parsing on Chinese Pennbank
11 code implementations • 27 Jul 2016 • Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, Jianfeng Gao
In this paper, we design a benchmark task and provide the associated datasets for recognizing face images and link them to corresponding entity keys in a knowledge base.
no code implementations • 15 Jun 2016 • Jianshu Chen, Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng
In particular, we show that with regularization via a generative model, learning with the proposed unsupervised objective function converges to an optimal solution.
1 code implementation • EMNLP 2016 • Ji He, Mari Ostendorf, Xiaodong He, Jianshu Chen, Jianfeng Gao, Lihong Li, Li Deng
We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space.
1 code implementation • NAACL 2016 • Ting-Hao, Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, Margaret Mitchell
We introduce the first dataset for sequential vision-to-language, and explore how this data may be used for the task of visual storytelling.
no code implementations • 6 Apr 2016 • Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James Allen
We created a new corpus of ~50k five-sentence commonsense stories, ROCStories, to enable this evaluation.
1 code implementation • EMNLP 2016 • David Golub, Xiaodong He
We show that a character-level encoder-decoder framework can be successfully applied to question answering with a structured knowledge base.
no code implementations • 30 Mar 2016 • Kenneth Tran, Xiaodong He, Lei Zhang, Jian Sun, Cornelia Carapcea, Chris Thrasher, Chris Buehler, Chris Sienkiewicz
We present an image caption system that addresses new challenges of automatically describing images in the wild.
2 code implementations • ACL 2016 • Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, Lucy Vanderwende
There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images.
no code implementations • 12 Jan 2016 • Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng
In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).
no code implementations • 19 Nov 2015 • Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng, Paul Smolensky
Question answering tasks have shown remarkable progress with distributed vector representation.
3 code implementations • ACL 2016 • Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, Mari Ostendorf
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games.
16 code implementations • CVPR 2016 • Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola
Thus, we develop a multiple-layer SAN in which we query an image multiple times to infer the answer progressively.
Ranked #5 on Visual Question Answering (VQA) on VQA v1 test-std
no code implementations • 10 Sep 2015 • Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li Deng, Ji He
Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states.
1 code implementation • NeurIPS 2015 • Jianshu Chen, Ji He, Yelong Shen, Lin Xiao, Xiaodong He, Jianfeng Gao, Xinying Song, Li Deng
We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model using Back Propagation (i. e., BP-sLDA), which maximizes the posterior probability of the prediction variable given the input document.
1 code implementation • WWW 2015 • Ali Elkahky, Yang song, Xiaodong He
We extend the model to jointly learn from features of items from different domains and user features by introducing a multi-view Deep Learning model.
no code implementations • IJCNLP 2015 • Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, Margaret Mitchell
Two recent approaches have achieved state-of-the-art results in image captioning.
no code implementations • 13 Apr 2015 • Xiaodong He, Rupesh Srivastava, Jianfeng Gao, Li Deng
The learned representations attempt to capture the combination of various visual concepts and cues.
no code implementations • 11 Apr 2015 • Yelong Shen, Ruoming Jin, Jianshu Chen, Xiaodong He, Jianfeng Gao, Li Deng
Co-occurrence Data is a common and important information source in many areas, such as the word co-occurrence in the sentences, friends co-occurrence in social networks and products co-occurrence in commercial transaction data, etc, which contains rich correlation and clustering information about the items.
no code implementations • 24 Feb 2015 • Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, Rabab Ward
The results show that the proposed method in this paper significantly outperforms it for web document retrieval task.
9 code implementations • 20 Dec 2014 • Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng
We consider learning representations of entities and relations in KBs using the neural-embedding approach.
Ranked #10 on Link Prediction on UMLS
1 code implementation • CVPR 2015 • Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollár, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick, Geoffrey Zweig
The language model learns from a set of over 400, 000 image descriptions to capture the statistics of word usage.
Ranked #1 on Image Captioning on COCO Captions test
no code implementations • 14 Nov 2014 • Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng
In this paper we present a unified framework for modeling multi-relational representations, scoring, and learning, and conduct an empirical study of several recent multi-relational embedding models under the framework.
no code implementations • 28 Nov 2013 • Jianfeng Gao, Xiaodong He, Wen-tau Yih, Li Deng
The results show that the new semantic-based phrase translation model significantly improves the performance of a state-of-the-art phrase-based statistical machine translation sys-tem, leading to a gain of 0. 7-1. 0 BLEU points.
5 code implementations • CIKM 2013 • Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck
The proposed deep structured semantic models are discriminatively trained by maximizing the conditional likelihood of the clicked documents given a query using the clickthrough data.