Towards Robust and Linguistically Informed Evaluation of LLMs: from benchmarks to Real-world applications (TRAIL-LLMs) (AIA2025-163322-C65)
Research project developed at Universidad Nacional de Educación a Distancia (UNED) by the Departamento de Lenguajes y Sistemas Informáticos of the Escuela Técnica Superior de Ingeniería Informática, in the framework of the nlp.uned.es group and the LENAR research center, and by the Departamento de Filologías Extranjeras y sus Lingüísticas of the Facultad de Filología in the framework of the ACTUALing group.
Funding
This project has been funded by the 2025 call «Proyectos de Investigación en el Ámbito de la Inteligencia Artificial» in the frame of the Programa Estatal para la Investigación y el Desarrollo Experimental, del Plan Estatal de Investigación Científica y Técnica y de Innovación 2024-2027.
Budget: 297.500 euros.
AIM AND METHODOLOGY
This project aims to evaluate large language models (LLMs) from two perspectives: assessing their core capabilities and evaluating their performance across three real-world application scenarios (use cases):
USE CASE 1. Plain language in the Administration-legal domain.
USE CASE 2. Data anonymisation for privacy protection.
USE CASE 3. The biomedical domain.
Specifically, the arText team is responsible for the USE CASE 1, for which the following methodology has been defined:
| METHODOLOGY | |||||
| STAGE 1 | STAGE 2 | STAGE 3 | STAGE 4 | STAGE 5 | STAGE 6 |
| Compilation of a corpus of texts from the administrative domain. | Adapting the corpus to plain language using LLMs. | Evaluation of LLMs performance to clarify the corpus. | Human post-editing of plain language adaptations. | Design and implementation of a hybrid system that combines generative and symbolic AI to generate plain texts. | Evaluation of the hybrid system’s performance. |
TEAM
From the nlp.uned.es group:
- Julio Gonzalo (principal investigator 1) (UNED)
- Enrique Amigó (UNED)
- Guillermo Marco (UNED)
- Anselmo Peñas (UNED)
- Laura Plaza (UNED)
- Álvaro Rodrigo (UNED)
- Eva Sánchez (UNED)
From the arText team:
- Iria da Cunha (principal investigator 2) (UNED)
- M. Ángeles Escobar (UNED)
- Montserrat Bermúdez Bausela (UNED)
- M. Amor Montané (Universitat de Barcelona)
- Susana Viñuales (UNED)
- Sara Pistola (UNED)
- Laia Vidal-Sabanés (Universitat de Barcelona)
coordinated project
The TRAIL-LLMs project is a subproject of the coordinated project ‘Humanising AI with Language Technology’ (HumanAIze), led by the Basque Center for Language Technology (HiTZ) at the Universidad del País Vasco (EHU), and in which the following organisations, in addition to the UNED, are participating:
- the Centro de Inteligencia Digital (CENID) at the Universidad de Alicante (UA);
- the Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS) at the Universidade de Santiago de Compostela (USC);
- the Centro de Estudios Avanzados en Tecnologías de la Información y de la Comunicación (CEATIC) at the Universidad de Jaén (UJA);
- and the Barcelona Supercomputing Center (BSC).
NEWS
Click here.