Evaluating LLMs at Evaluating Temporal Generalization

1 code implementation • 14 May 2024 • Chenghao Zhu, Nuo Chen, Yufei Gao, Benyou Wang

The rapid advancement of Large Language Models (LLMs) highlights the urgent need for evolving evaluation methodologies that keep pace with improvements in language comprehension and information processing.

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

1 code implementation • 23 Nov 2023 • Wentao Ge, Shunian Chen, Guiming Hardy Chen, Zhihong Chen, Junying Chen, Shuo Yan, Chenghao Zhu, Ziyue Lin, Wenya Xie, Xinyi Zhang, Yichen Chai, Xiaoyu Liu, Dingjie Song, Xidong Wang, Anningzhe Gao, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang

Multimodal large language models (MLLMs) (e. g., GPT-4V, LLaVA, and Claude-3) have broadened the scope of AI applications.

XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

no code implementations • 15 Aug 2019 • Pengyuan Liu, Yuning Deng, Chenghao Zhu, Han Hu

Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used.

Machine Reading Comprehension Sentence

Emotion Action Detection and Emotion Inference: the Task and Dataset

1 code implementation • 16 Mar 2019 • Pengyuan Liu, Chengyu Du, Shuofeng Zhao, Chenghao Zhu

We are currently releasing the data-set with 10, 603 samples and 15, 892 events, basic statistic analysis and baseline on both emotion causality and emotion inference tasks.

Action Detection Emotion Classification +2