Model-based Algorithmic Auditing of Social Media AI Algorithms

This position paper introduces a novel paradigm for oversight of social media AI algorithms, called the model-based algorithmic auditing. In general, the algorithmic auditing is a process of automated dynamic black-box assessment of real-world software system behavior. In a so-called sockpuppeting audit, impostor bots stimulate the platform with a simulated user behavior and observe the responses of the audited system (e.g., a recommender system). Algorithmic auditing is able to disclose interesting traits of AI systems (e.g., biases), which may be otherwise opaque. However, technical and methodological difficulties make audits costly, hard to reproduce, and hard to transfer cross-platform and cross-domain. To overcome this, the model-based algorithmic auditing introduces a platform-agnostic social media model which provides a simplified and aggregated representation of users, content, and interactions between them. The model supports or even automates challenging steps of the audit, like assisting human experts in creation of abstract audit scenarios, or predicting next user interactions. The reduction of manual effort makes the auditing more representative, cross-platform, and longitudinal, ultimately enabling more efficient oversight of social media algorithms by regulators, auditors and other stakeholders.

Cite: Srba, I., Pecher, B., Simko, J., Moro, R. & Bielikova, M.. (2025). Model-based Algorithmic Auditing of Social Media AI Algorithms. Proceedings of Fourth European Workshop on Algorithmic Fairness, in Proceedings of Machine Learning Research 294:439-445 Available from https://proceedings.mlr.press/v294/srba25a.html.