Jun 9. 2026

Slovak NLP Community Meeting #4

Program

12:30 Opening of the academic session
12:30 Roundtable discussion (academic partners)
13:30 Coffee break, arrival of industry partners
14:00 Opening of the joint session (academic and industrial partners)
14:00 Introduction to the SK NLP community
14:30 Model benchmarking: Towards Comprehensive Evaluation of Language Models for Slovak (Andrej Ridzik et al.)
15:00 End of the online session and coffee break (in parallel with the rest of the program)
15:00 Poster session (in-person only):
— Research section – projects and PhD students
— Industry section
(cca 16:30) Closing

Towards Comprehensive Evaluation of Language Models for Slovak

Evaluating language models for Slovak has largely relied on supervised benchmarks, where models are fine-tuned and tested on task-specific data. However, a growing class of applications — retrieval-augmented generation, semantic search, document clustering — depends on zero-shot embedding quality, for which no Slovak benchmark existed.

In this talk, we will present SkMTEB, our recently accepted ACL 2026 paper introducing the first comprehensive Slovak text embedding benchmark. We will cover the benchmark design, key findings from our model evaluation, and the development of compact open-weight Slovak embedding models. Finally, we will outline our ongoing efforts towards broader evaluation of large language models on Slovak, motivating further research in this direction.

Poster Session

with names of presenting (co)authors

Research track – projects

  1. SlovakCOMET: An Open, Reproducible Quality-Estimation Metric for English – Slovak Machine Translation, Marek Šuppa, Jaroslav Kopčan, et al. (FMFI UK, NLP KInIT, et al.)
  2. MultiCW: A Large-Scale Balanced Benchmark Dataset for Training Robust Check-Worthiness Detection Models (project DisTraceAI), Róbert Móro, et al. (WUDAP KInIT)
  3. LLMs as Generators and Evaluators (projects GEPERO and RobIndAI), Dominik Macko, et al. (WUDAP KInIT)
  4. Corpora of Ľ. Štúr Institute of Linguistics, Ivor Uhliarik (JULS SAV)
  5. Projects of the Department of the Speech Analysis Synthesis of Institute of informatics SAS, Viktória Kevická (UI SAV)
  6. RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar, Marek Šuppa, Viktória Ondrejová (FMFI UK, Cisco)

Research track – PhD students

  1. Assessing Web Search Credibility and Response Groundedness in Chat Assistants, Ivan Vykopal (NLP KInIT)
  2. PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models, Róbert Belanec (WUDAP KInIT)
  3. Interpretable Predictability-Based AI Text Detection: A Replication Study, Adam Škurla (WUDAP KInIT)
  4. AI-generated text detection on HC3 dataset, Viliam Balara (UUI TUKE)
  5. Hybrid Transformer Ensemble for Toxic Speech Detection in Spoken Transcripts, Peter Demeter (UUI TUKE)
  6. Distillation-guided Selection of LoRA Target Modules Using Checkpoint Delta Norms, Samuel Fodor (UUI TUKE)
  7. Early Alzheimer’s Detection Using Siamese Conformer Networks on Slovak Confrontation Naming Tasks, Tomáš Černáček (UI SAV)
  8. Zero-shot LLM-as-a-Judge Prediction of the Machine Translation Quality from English into Slovak, Martin Chotár (NLP Lab UKF)
  9. Evaluating the Quality of English-Slovak Translations using Large Language Models, Matúš Kleštinec (NLP Lab UKF)

Industry track

  1. SLAIH: Slovak Language AI Hub, Andrej Ridzik, Marián Šimko, et al. (NLP KInIT)
  2. SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation, Marek Šuppa, Andrej Ridzik, Natália Kňažeková, et al. (FMFI UK, NLP KInIT, et al.)
  3. Promptillery: From Prompts to Production via Iterative LLM Distillation, Marek Šuppa, Jaroslav Kopčan, et al. (FMFI UK, NLP KInIT)
  4. Slovak Speech Recognition Benchmark: Open-Weight vs. Commercial Models and the Role of Fine-Tuning, Soňa Dulíková, et al. (KInIT), TBA (Telekom)
  5. Specification Agent: Human-in-the-Loop AI for Software Issue Specification, Alexander Brecko, et al. (KInIT), Martin Čambal, et al. (ESET)
  6. Legal Document Classification and Receivables Lifecycle Modeling: An Industry Collaboration with Bencont, Marcel Veselý, et al. (KInIT), Ondrej Gutten, et al. (Bencont)
  7. Practical applications of the Department of the Speech Analysis Synthesis of Institute of informatics SAS, Marián Trnka (UI SAV)
  8. Adapting Open ASR Models for improved Slovak Speech Recognition, Martin Tamajka, Roman Matúš, et al. (NLP KInIT)

More information about the SK NLP community.

Supporters

The visit was organised in collaboration with the lorAI project, one of the goals of which is to attend, contribute to, or organise events aimed at promoting science among young people and the general public.