no code implementations • 16 Jan 2023 • Youmna Farag, Charlotte O. Brand, Jacopo Amidei, Paul Piwek, Tom Stafford, Svetlana Stoyanchev, Andreas Vlachos
We show that while both models perform closely in terms of opening up minds, the argument-based model is significantly better on other dialogue properties such as engagement and clarity.
no code implementations • COLING 2020 • Jacopo Amidei, Paul Piwek, Alistair Willis
Our interpretation of IRT offers an original bias identification method that can be used to compare annotators{'} bias and characterise annotation disagreement.
no code implementations • WS 2019 • Jacopo Amidei, Paul Piwek, Alistair Willis
Rating and Likert scales are widely used in evaluation experiments to measure the quality of Natural Language Generation (NLG) systems.
no code implementations • WS 2019 • Jacopo Amidei, Paul Piwek, Alistair Willis
Following Sampson and Babarczy (2008), Lommel et al. (2014), Joshi et al. (2016) and Amidei et al. (2018b), such phenomena can be explained in terms of irreducible human language variability.
no code implementations • WS 2018 • Jacopo Amidei, Paul Piwek, Alistair Willis
In the last few years Automatic Question Generation (AQG) has attracted increasing interest.
no code implementations • COLING 2018 • Jacopo Amidei, Paul Piwek, Alistair Willis
For this reason, we believe that annotation schemes for natural language generation tasks that are aimed at evaluating language quality need to be treated with great care.
no code implementations • 18 May 2018 • Pasquale Iero, Allan Third, Paul Piwek
This paper describes a formalism that subsumes Peterson's intermediate quantifier syllogistic system, and extends the ideas by van Eijck on Aristotle's logic.
no code implementations • WS 2017 • Richard Doust, Paul Piwek
Most work on automatic generation of narratives, and more specifically suspenseful narrative, has focused on detailed domain-specific modelling of character psychology and plot structure.
no code implementations • COLING 2016 • Brian Pl{\"u}ss, Paul Piwek
In contrast, we start from rules for normal/correct dialogue behaviour - i. e., a dialogue game - which in principle can be derived from a corpus of cooperative dialogues, and provide a quantitative measure for the degree to which participants comply with these rules.