Slovak NLP Community Meeting #2

12:30 Opening
12:30 Marek Šuppa et al. (UK, KInIT, TUKE): skLEP: Slovak GLUE Benchmark
13:00 Martin Fajčík (FIT BUT Brno): BenCzechMark and Beyond
13:30 Coffee break
13:45 Invited talk: Michal Štefánik (University of Helsinki): Robust Language Models and Where to Find Them*
14:30 Coffee break
14:45 Miroslav Blšták (KInIT, Eduself): Terminology Dictionary for Slovak language
15:15 Roundtable (open-ended)
16:30+ Closing

*Part of the program was supported by the slovaks.ai initiative.

Language models (LMs) have emerged into a technology adopted in a wide variety of use-cases, today largely exceeding traditional NLP tasks. Despite that, over the last years, we have made little progress in LMs’ applicability in tasks with limited data and tasks requiring reliable decision making, bearing huge potential for automation. Both these limitations can be attributed to models’ limited robustness in out-of-distribution settings.

In this talk, I will share our experience with making language models more robust across languages, domains and tasks, and underline some general rules for improving robustness. Finally, I will outline our vision for achieving progress beyond the inherent limitations of the Transformer architecture, motivating further research in several key directions.

The second community meeting was again held at KInIT, in the premises of The Spot at SkyPark Offices in Bratislava. The program focused on benchmarking. Marek Šuppa first presented the results of the benchmarking working group (including a scientific paper accepted at ACL 2025!) and invitation was also accepted by Martin Fajčík from FIT BUT Brno, who shared experience with benchmarking language models for Czech. A special highlight of the program, supported by the slovaks.ai initiative, was the invited talk by Michal Štefánik from University of Helsinki. Miroslav Blšták from KInIT presented a newly developed terminology dictionary for the Slovak language. During the open-ended roundtable, individual research teams shared updates on their ongoing work.