Report on the Current State of Societal Biases in Slovak AI

Pikuliak, M., Oreško, Š., Burda, K., Gavorník, A., Mesarčík, M., Hrčková, A., Kottulová, J., Szapuová, M., Podroužek, J., Šimko, M.

This report serves as a concluding report of our project Societal Biases in Slovak AI running from November 2022 to October 2023. The goal of our project was to understand how gender biases in particular affect Slovak AI systems, but also to increase the awareness of this issue in both the expert community and the general public, as this is an issue that affects virtually everyone using modern communication technologies. Our aim was to approach this issue from an interdisciplinary point of view. The team included people with technical expertise in AI and natural language processing; but also AI ethics experts, social scientists, translators, and gender experts.

The main contribution of our project is the evaluation of multiple types of AI systems, namely English-to-Slovak machine translation systems, Slovak masked language models, and Slovak speech-to-text systems. We have proposed and implemented an evaluation methodology and observed whether these systems exhibit biased behavior with respect to gender. Each experiment was aimed at a specific type of biased behavior, such as male as norm behavior, stereotypical thinking, or non-equitable performance. Worryingly, we were able to discover some sort of problematic behavior in all the models taken into consideration

Cite: Pikuliak, M., Oreško, Š., Burda, K., Gavorník, A., Mesarčík, M., Hrčková, A., Kottulová, J., Szapuová, M., Podroužek, J., & Šimko, M. (2023). Report on the Current State of Societal Biases in Slovak AI. Zenodo. https://doi.org/10.5281/zenodo.10059075.

Autori

Matúš Pikuliak
Research Consultant 10/2022-01/2024
Viac
Štefan Oreško
Researcher 03/2022-10/2024
Viac
Kamil Burda
Researcher
Viac
Adrián Gavorník
Research Intern
Viac
Matúš Mesarčík
Ethics and Law Specialist
Viac
Andrea Hrčková
Researcher
Viac
Juraj Podroužek
Lead and Researcher
Viac
Marián Šimko
Lead and Researcher
Viac