Slovak Telekom: Independent benchmark for choosing the right Slovak speech-to-text model

Call center analysis tools like summarization, topic classification, and sentiment analysis all depend on one thing: a reliable speech-to-text model. But which one should a company choose for Slovak? Until now, there was no way to answer this with data. No benchmark of speech-to-text models existed for the Slovak language, since public leaderboards focus on English or on multilingual datasets where Slovak is largely absent. Without objective evidence, choosing between commercial providers and open-weight models carries significant financial and operational risk.

The goal of the project was to close this gap: build an independent benchmark of speech-to-text models for Slovak, with a special focus on the telecommunications domain, and give Slovak Telekom a solid evidence base for choosing the right solution.

Building such a benchmark is out of reach for most individual companies. It requires hundreds of hours of curated Slovak audio with verified transcripts, a fair evaluation methodology, and the infrastructure to run dozens of models over the whole dataset.

We collected and curated 354 hours of Slovak audio with transcripts across multiple domains, combining public datasets, data provided by Slovak Telekom, and new datasets created at KInIT, including municipal council hearings, read literary texts, and scripted call center dialogues. On this data, we evaluated more than 20 speech-to-text models, both commercial services and open-weight models, from several perspectives that matter in real applications:

The main output is a detailed benchmark report that gives Slovak Telekom an objective comparison of available speech-to-text solutions for Slovak. The results serve as a concrete performance baseline that any current or future provider can be measured against, turning vendor selection from a matter of trust into a matter of evidence. The benchmark is also built as a reusable framework: adding a new model requires no methodological changes, so the evaluation stays current as the market evolves.

The project also opened promising follow-up directions. We proposed a process for labeling Slovak Telekom’s anonymized call center recordings by combining multiple strong models with human validation on a sample, producing in-domain data at a fraction of the cost of manual transcription. Such data can be used to fine-tune open-weight models for the telco domain and an end-to-end evaluation of the whole analysis pipeline, from audio to insights.