no code implementations • NAACL (ACL) 2022 • Gongzheng li, Yadong Xi, Jingzhen Ding, Duan Wang, Ziyang Luo, Rongsheng Zhang, Bai Liu, Changjie Fan, Xiaoxi Mao, Zeng Zhao
To fill such a gap, we introduce a scalable inference solution: Easy and Efficient Transformer (EET), including a series of transformer inference optimization at the algorithm and implementation levels.
no code implementations • ACL 2022 • Le Zhang, Rongsheng Zhang, Xiaoxi Mao, Yongzhu Chang
In this paper, we demonstrate the QiuNiu, a Chinese lyrics generation system which is conditioned on passage-level text rather than a few attributes or keywords.
1 code implementation • WWW 2022 • Jiashu Pu, Jianshi Lin, Xiaoxi Mao, Jianrong Tao, Xudong Shen, Yue Shang, Runze Wu
Players of online games generate rich behavioral data during gaming.
no code implementations • Findings (NAACL) 2022 • Ziyang Luo, Yadong Xi, Jing Ma, Zhiwei Yang, Xiaoxi Mao, Changjie Fan, Rongsheng Zhang
In contrast, Transformer Decoder with the causal attention masks is naturally sensitive to the word order.
no code implementations • EMNLP 2020 • Rongsheng Zhang, Xiaoxi Mao, Le Li, Lin Jiang, Lin Chen, Zhiwei Hu, Yadong Xi, Changjie Fan, Minlie Huang
In the lyrics generation process, \textit{Youling} supports traditional one pass full-text generation mode as well as an interactive generation mode, which allows users to select the satisfactory sentences from generated candidates conditioned on preceding context.
no code implementations • 18 Jan 2022 • Jiashu Pu, Guandan Chen, Yongzhu Chang, Xiaoxi Mao
Existing task-oriented chatbots heavily rely on spoken language understanding (SLU) systems to determine a user's utterance's intent and other key information for fulfilling specific tasks.
no code implementations • 16 Dec 2021 • Yadong Xi, Jiashu Pu, Xiaoxi Mao
The wave of pre-training language models has been continuously improving the quality of the machine-generated conversations, however, some of the generated responses still suffer from excessive repetition, sometimes repeating words from utterance, sometimes repeating words within self-generated responses, or both.
no code implementations • 1 Nov 2021 • Rongsheng Zhang, Yinhe Zheng, Xiaoxi Mao, Minlie Huang
However, fine-tuning all the parameters of the PrLM on a small domain-specific corpus distort the learned generic knowledge, and it is also expensive to deployment a whole fine-tuned PrLM for each domain.
no code implementations • 29 Sep 2021 • Ziyang Luo, Yadong Xi, Jing Ma, Xiaoxi Mao, Changjie Fan
A common limitation of Transformer Encoder's self-attention mechanism is that it cannot automatically capture the information of word order, so one needs to feed the explicit position encodings into the target model.
1 code implementation • EMNLP 2021 • Chen Henry Wu, Yinhe Zheng, Xiaoxi Mao, Minlie Huang
Grounded dialogue models generate responses that are grounded on certain concepts.
2 code implementations • 30 Aug 2021 • Jian Guan, Zhuoer Feng, Yamei Chen, Ruilin He, Xiaoxi Mao, Changjie Fan, Minlie Huang
Therefore, we propose a story-centric benchmark named LOT for evaluating Chinese long text modeling, which aggregates two understanding tasks and two generation tasks.
no code implementations • ACL 2021 • Yadong Xi, Xiaoxi Mao, Le Li, Lei Lin, Yanjiang Chen, Shuhan Yang, Xuhan Chen, Kailun Tao, Zhi Li, Gongzheng li, Lin Jiang, Siyan Liu, Zeng Zhao, Minlie Huang, Changjie Fan, Zhipeng Hu
Equipped with GPT-2 and the latest GPT-3, AI Dungeon has been seen as a famous example of the powerful text generation capabilities of large-scale pre-trained language models, and a possibility for future games.
1 code implementation • ACL 2021 • Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, Minlie Huang
Generating long and coherent text is an important but challenging task, particularly for open-ended language generation tasks such as story generation.
1 code implementation • ACL 2021 • Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, Minlie Huang
Automatic metrics are essential for developing natural language generation (NLG) models, particularly for open-ended language generation tasks such as story generation.
1 code implementation • 26 Apr 2021 • Gongzheng li, Yadong Xi, Jingzhen Ding, Duan Wang, Bai Liu, Changjie Fan, Xiaoxi Mao, Zeng Zhao
To fill such a gap, we introduce a scalable inference solution: Easy and Efficient Transformer (EET), including a series of transformer inference optimization at the algorithm and implementation levels.
no code implementations • ACL 2021 • Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers.
1 code implementation • 27 Sep 2020 • Yinhe Zheng, Zikai Chen, Rongsheng Zhang, Shilei Huang, Xiaoxi Mao, Minlie Huang
However, this task is far from well-explored due to the difficulties of rendering a particular style in coherent responses, especially when the target style is embedded only in unpaired texts that cannot be directly used to train the dialogue model.
1 code implementation • EMNLP 2020 • Rongsheng Zhang, Yinhe Zheng, Jianzhi Shao, Xiaoxi Mao, Yadong Xi, Minlie Huang
Further, a model-level distillation process is employed to distill a teacher model trained on high-quality paired data to augmented dialogue pairs, thereby preventing dialogue models from being affected by the noise in the augmented data.
2 code implementations • 12 Nov 2019 • Yinhe Zheng, Rongsheng Zhang, Xiaoxi Mao, Minlie Huang
Further, to incorporate the target persona in the decoding process and to balance its contribution, an attention routing structure is devised in the decoder to merge features extracted from the target persona and dialogue contexts using dynamically predicted weights.