publications

2024

1. Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs (Best Non Publicized Paper Award 🎖️)

   Simone Balloccu, Patrı́cia Schmidtová, Mateusz Lango, and 1 more author

   In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Mar 2024

   Abs

   Natural Language Processing (NLP) research is increasingly focusing on the use of Large Language Models (LLMs), with some of the most popular ones being either fully or partially closed-source. The lack of access to model details, especially regarding training data, has repeatedly raised concerns about data contamination among researchers. Several attempts have been made to address this issue, but they are limited to anecdotal evidence and trial and error. Additionally, they overlook the problem of indirect data leaking, where modelsare iteratively improved by using data coming from users. In this work, we conduct the first systematic analysis of work using OpenAI’s GPT-3.5 and GPT-4, the most prominently used LLMs today, in the context of data contamination. By analysing 255 papers and considering OpenAI’s data usage policy, we extensively document the amount of data leaked to these models during the first year after the model’s release. We report that these models have been globally exposed to ∼4.7M samples from 263 benchmarks. At the same time, we document a number of evaluation malpractices emerging in the reviewed papers, such as unfair or missing baseline comparisons and reproducibility issues. We release our results as a collaborative project on https://leak-llm.github.io/, where other researchers can contribute to our efforts.

2. ReproHum #0043-4: Evaluating Summarization Models: investigating the impact of education and language proficiency on reproducibility

   Mateusz Lango, Patricia Schmidtova, Simone Balloccu, and 1 more author

   In Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024, May 2024

   Abs

   In this paper, we describe several reproductions of a human evaluation experiment measuring the quality of automatic dialogue summarization (Feng et al., 2021). We investigate the impact of the annotators’ highest level of education, field of study, and native language on the evaluation of the informativeness of the summary. We find that the evaluation is relatively consistent regardless of these factors, but the biggest impact seems to be a prior specific background in natural language processing (as opposed to, e.g. a background in computer sci- ence). We also find that the experiment setup (asking for single vs. multiple criteria) may have an impact on the results.

3. factgenie: A Framework for Span-based Evaluation of Generated Texts

   Zdeněk Kasner, Ondrej Platek, Patricia Schmidtova, and 2 more authors

   In Proceedings of the 17th International Natural Language Generation Conference: System Demonstrations, Sep 2024

   Abs

   We present ‘factgenie‘: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccuracies or irrelevant text. With ‘factgenie‘, the annotations can be collected both from human crowdworkers and large language models. Our framework consists of a web interface for data visualization and gathering text annotations, powered by an easily extensible codebase.

4. Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices (Best Evaluation Paper Award 🎖️)

   Patricia Schmidtova, Saad Mahamood, Simone Balloccu, and 6 more authors

   In Proceedings of the 17th International Natural Language Generation Conference, Sep 2024

   Abs

   Automatic metrics are extensively used to evaluate Natural Language Processing systems. However, there has been increasing focus on how the are used and reported by practitioners within the field. In this paper, we have conducted a survey on the use of automatic metrics, focusing particularly on natural language generation tasks. We inspect which metrics are used as well as why they are chosen and how their use is reported. Our findings from this survey reveal significant shortcomings, including inappropriate metric usage, lack of implementation details and missing correlations with human judgements. We conclude with recommendations that we believe authors should follow to enable more rigour within the field.

5. Ask the experts: sourcing a high-quality nutrition counseling dataset through Human-AI collaboration

   Simone Balloccu, Ehud Reiter, Karen Jia-Hui Li, and 5 more authors

   In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

   Abs

   Large Language Models (LLMs) are being employed by end-users for various tasks, including sensitive ones such as health counseling, disregarding potential safety concerns. It is thus necessary to understand how adequately LLMs perform in such domains. We conduct a case study on ChatGPT in nutrition counseling, a popular use-case where the model supports a user with their dietary struggles. We crowd-source real-world diet-related struggles, then work with nutrition experts to generate supportive text using ChatGPT. Finally, experts evaluate the safety and text quality of ChatGPT’s output. The result is the HAI-coaching dataset, containing ~2.4K crowdsourced dietary struggles and ~97K corresponding ChatGPT-generated and expert-annotated supportive texts. We analyse ChatGPT’s performance, discovering potentially harmful behaviours, especially for sensitive topics like mental health. Finally, we use HAI-coaching to test open LLMs on various downstream tasks, showing that even the latest models struggle to achieve good performance. HAI-coaching is available at https://github.com/uccollab/hai-coaching/

2023

1. Creation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues

   Zixiu Wu, Simone Balloccu, Vivek Kumar, and 3 more authors

   Future Internet, Mar 2023
2. Smart Selection of Useful Insights from Wearables

   Allmin Susaiyah, Aki Härmä, Simone Balloccu, and 2 more authors

   In 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), Jun 2023
3. Are Experts Needed? On Human Evaluation of Counselling Reflection Generation

   Zixiu Wu, Simone Balloccu, Ehud Reiter, and 3 more authors

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs

   Reflection is a crucial counselling skill where the therapist conveys to the client their interpretation of what the client said. Language models have recently been used to generate reflections automatically, but human evaluation is challenging, particularly due to the cost of hiring experts. Laypeople-based evaluation is less expensive and easier to scale, but its quality is unknown for reflections. Therefore, we explore whether laypeople can be an alternative to experts in evaluating a fundamental quality aspect: coherence and context-consistency. We do so by asking a group of laypeople and a group of experts to annotate both synthetic reflections and human reflections from actual therapists. We find that both laypeople and experts are reliable annotators and that they have moderate-to-strong inter-group correlation, which shows that laypeople can be trusted for such evaluations. We also discover that GPT-3 mostly produces coherent and consistent reflections, and we explore changes in evaluation results when the source of synthetic reflections changes to GPT-3 from the less powerful GPT-2.

2022

1. Data Augmentation for Reliability and Fairness in Counselling Quality Classification

   Vivek Kumar, Simone Balloccu, Zixiu Wu, and 4 more authors

   In Proceedings of the 1st Workshop on Scarce Data in Artificial Intelligence for Healthcare - SDAIH @ IJCAI 2022, Jul 2022
2. Beyond calories: evaluating how tailored communication reduces emotional load in diet-coaching

   Simone Balloccu, and Ehud Reiter

   In Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval), May 2022

   Abs

   Dieting is a behaviour change task that is difficult for many people to conduct successfully. This is due to many factors, including stress and cost. Mobile applications offer an alternative to traditional coaching. However, previous work on apps evaluation only focused on dietary outcomes, ignoring users’ emotional state despite its influence on eating habits. In this work, we introduce a novel evaluation of the effects that tailored communication can have on the emotional load of dieting. We implement this by augmenting a traditional diet-app with affective NLG, text-tailoring and persuasive communication techniques. We then run a short 2-weeks experiment and check dietary outcomes, user feedback of produced text and, most importantly, its impact on emotional state, through PANAS questionnaire. Results show that tailored communication significantly improved users’ emotional state, compared to an app-only control group.

3. Anno-MI: A Dataset of Expert-Annotated Counselling Dialogues

   Zixiu Wu, Simone Balloccu, Vivek Kumar, and 4 more authors

   In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2022
4. Comparing informativeness of an NLG chatbot vs graphical app in diet-information domain

   Simone Balloccu, and Ehud Reiter

   In Proceedings of the 15th International Conference on Natural Language Generation, Jul 2022

   Abs

   Visual representation of data like charts and tables can be challenging to understand for readers. Previous work showed that combining visualisations with text can improve the communication of insights in static contexts, but little is known about interactive ones. In this work we present an NLG chatbot that processes natural language queries and provides insights through a combination of charts and text. We apply it to nutrition, a domain communication quality is critical. Through crowd-sourced evaluation we compare the informativeness of our chatbot against traditional, static diet-apps. We find that the conversational context significantly improved users’ understanding of dietary data in various tasks, and that users considered the chatbot as more useful and quick to use than traditional apps.

5. Towards In-Context Non-Expert Evaluation of Reflection Generation for Counselling Conversations

   Zixiu Wu, Simone Balloccu, Rim Helaoui, and 2 more authors

   In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), Dec 2022

   Abs

   Reflection is an essential counselling strategy, where the therapist listens actively and responds with their own interpretation of the client’s words. Recent work leveraged pre-trained language models (PLMs) to approach reflection generation as a promising tool to aid counsellor training. However, those studies used limited dialogue context for modelling and simplistic error analysis for human evaluation. In this work, we take the first step towards addressing those limitations. First, we fine-tune PLMs on longer dialogue contexts for reflection generation. Then, we collect free-text error descriptions from non-experts about generated reflections, identify common patterns among them, and accordingly establish discrete error categories using thematic analysis. Based on this scheme, we plan for future work a mass non-expert error annotation phase for generated reflections followed by an expert-based validation phase, namely “whether a coherent and consistent response is a good reflection”.

2021

1. Unaddressed challenges in persuasive dieting chatbots

   Simone Balloccu, Ehud Reiter, Matteo G. Collu, and 3 more authors

   In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, Dec 2021

   Abs

   Diet coaching is a behaviour change task which requires lots of interaction with patients. E-health apps gathered lots of interest in research and, recently, chatbots have been leveraged to address this task, with a focus on persuasion to motivate people towards behaviour change. In this paper, we take a look at current approaches in building persuasive dieting chatbots and expose a number of major unsolved challenges. We motivate them with evidence from previous work and show that current chatbots don’t approach certain scenarios properly, hence limiting their communication and persuasion capabilities.

2020

1. A NLG Framework for User Tailoring and Profiling in Healthcare.

   Simone Balloccu, Steffen Pauws, and Ehud Reiter

   In 1st Workshop on Smart Personal Health Interfaces (SmartPhil)@IUI, Mar 2020
2. Fully-unsupervised embeddings-based hypernym discovery

   Maurizio Atzori, and Simone Balloccu

   Information, May 2020

   Abs

   The hypernymy relation is the one occurring between an instance term and its general term (e.g., “lion” and “animal”, “Italy” and “country”). This paper we addresses Hypernym Discovery, the NLP task that aims at finding valid hypernyms from words in a given text, proposing HyperRank, an unsupervised approach that therefore does not require manually-labeled training sets as most approaches in the literature. The proposed algorithm exploits the cosine distance of points in the vector space of word embeddings, as already proposed by previous state of the art approaches, but the ranking is then corrected by also weighting word frequencies and the absolute level of similarity, which is expected to be similar when measuring co-hyponyms and their common hypernym. This brings us two major advantages over other approaches—(1) we correct the inadequacy of semantic similarity which is known to cause a significant performance drop and (2) we take into accounts multiple words if provided, allowing to find common hypernyms for a set of co-hyponyms—a task ignored in other systems but very useful when coupled with set expansion (that finds co-hyponyms automatically). We then evaluate HyperRank against the SemEval 2018 Hypernym Discovery task and show that, regardless of the language or domain, our algorithm significantly outperforms all the existing unsupervised algorithms and some supervised ones as well. We also evaluate the algorithm on a new dataset to measure the improvements when finding hypernyms for sets of words instead of singletons.

3. How are you? Introducing stress-based text tailoring

   Simone Balloccu, Ehud Reiter, Alexandra Johnstone, and 1 more author

   In Proceedings of the Workshop on Intelligent Information Processing and Natural Language Generation (IntelLanG)@ECAI 2020, Sep 2020

2019

1. OKgraph: Unsupervised Structured Data Extraction from Plain Text.

   Maurizio Atzori, Simone Balloccu, Andrea Bellanti, and 2 more authors

   In 10th Italian Information Retrieval Workshop (IIR 2019), Sep 2019

2018

1. Unsupervised singleton expansion from free text

   Maurizio Atzori, Simone Balloccu, and Andrea Bellanti

   In 2018 IEEE 12th International Conference on Semantic Computing (ICSC), Jan 2018