The 2nd LLMs4Subjects Shared Task: LLM-based Subject Tagging for the TIB
Technical Library's Open-Access Catalog

Theme: The Development of Energy- and Compute-Efficient LLM Systems

Organized as part of the German Evaluation (GermEval 2025) Shared Task
Series

10. - 12. September, 2025

Hildesheim, Germany

(co-located with KONVENS 2025 - Conference on Natural Language Processing)



2nd LLMs4Subjects Shared Task:
https://sites.google.com/view/llms4subjects-germeval/

KONVENS 2025: https://konvens-2025.hs-hannover.de/about/





Task Overview

LLMs4Subjects challenges the research community to develop cutting-edge
LLM-based solutions for subject tagging of technical records from Leibniz
University’s Technical Library (TIBKAT). Participants are tasked with
leveraging large language models (LLMs) to tag technical records using the
GND taxonomy. The task involves bilingual language modeling, as systems
must process technical documents in both German and English. Successful
solutions may be integrated into the operational workflows of TIB, the
Leibniz Information Centre for Science and Technology.

With the rapid advancements in LLMs, the focus is shifting toward making
these models more energy- and compute-efficient while maintaining high
performance. Recent innovations, such as the DeepSeek series, have
demonstrated how techniques like mixture-of-experts (MoE) and model
distillation can significantly reduce computational costs without
sacrificing effectiveness.

The 2nd LLMs4Subjects shared task highlights the importance of efficiency
in LLMs, encouraging participants to explore strategies that enhance model
performance while optimizing for energy consumption and inference speed. We
welcome approaches (but not limited to) that leverage model compression,
quantization, efficient fine-tuning, and adaptive computation techniques to
push the boundaries of sustainable AI development.

Subtasks

The 2nd LLMs4Subjects shared task organizes the following two subtasks:

Subtask 1 - Multi-Domain Classification of Library Records

Subtask 2 - Large-scale Multilabel Subject Indexing of Library Records


Important Dates
·       Release of training data: March 8, 2025 ·       Release of testing
data: May 23, 2025·       Deadline for system submissions: June 2, 2025·
    Evaluation end: June 27, 2025 ·       Paper submission deadline: July
7, 2025 ·       Notification of acceptance: June 28, 2025 ·       Camera-ready
paper due: August 15, 2025 ·       Workshop/KONVENS: September 10 - 12,
2025 (TBA)