The Brittle Compass: Navigating LLM Prompt Sensitivity in Slovak Migration Media Discourse

Harvan, S., Kopčan, J., Šuppa, M.^1,2, Findor, A.¹

¹ Cisco Systems, Slovakia, Bratislava
² Comenius University, Slovakia, Bratislava

In this work, we present a case study that explores various tasks centered around the topic of migration in Slovak, a low-resource language, such as topic relevance and geographical relevance classification, and migration source/destination location term extraction. Our results demonstrate that native (Slovak) prompts yield a modest, task-dependent gains, while large models show significant robustness to prompt variations compared to their smaller counterparts. Analysis reveals that instructions (system or task) emerge as the most critical prompt component, more so than the examples sections, with task-specific performance benefits being more pronounced than overall language effects.

Cite: Harvan, S., Kopčan, J., Šuppa, M., Findor, A. The Brittle Compass: Navigating LLM Prompt Sensitivity in Slovak Migration Media Discourse. In Proceedings of the First Workshop on Advancing NLP for Low-Resource Languages. (2025).