Facebook papers and disinformation

Towards the end of October 2021, various media outlets started to publish a series of investigative articles on problematic Facebook practices. These have become familiar under the names Facebook Files or Facebook Papers.

To get a better sense of the context, we must travel back to May 2017, when The Guardian published a series of articles titled “Facebook files”. They reported on a series of whistleblower accounts of their experience with working for Facebook (now Meta, but in this article we will refer to Facebook) as content  moderators. 

They reported on poor working conditions, insufficient training and a variety of psychological traumas caused by being exposed to extreme harmful imagery during the moderating process. Such negative experience serves as a foreshadowing of what was coming next and underlines the broader struggle with content moderation that Facebook had. These issues were further amplified  by the much larger Cambridge Analytica scandal, which fully uncovered how social media can be misused for political gains.

Fast forward to September 2021, when we are again faced with another controversy regarding Facebook practices. This time, it was The Wall Street Journal which first started publishing a series of investigative articles under the title “The Facebook Files”. The core consisted of 16 articles which were based on leaked internal documents and research data from Facebook. At the time of the publication, there were only speculations on how these documents were leaked. However, shortly after, on October 3rd, Frances Haugen, a former Facebook employee came out as the responsible whistleblower: 

The leaked documents, collectively named as Facebook Papers then laid at the core of the controversies and included internal presentations, research studies, discussions threads and strategy memos. These were made available to a consortium of international news agencies who started reporting on the problematic practices of Facebook.

The scope of the issues brought up by the leaked documents is broad. From the alleged inability of Facebook to moderate problematic content in certain languages or countries, to alleged use of Facebook by criminal groups for human trafficking. However, here we turn our focus to issues that relate to the spreading of misinformation and algorithmic content moderation.

One of the most significant findings was that according to the available documents, Facebook seemed to have created a sort of “VIP”  tier of its users – the so-called XCheck. It was originally intended as a quality control measure for high profile accounts such as celebrities, politicians and journalists to make sure that no unnecessary negative media attention would emerge from actions taken against them. However, what eventually ended up happening was that a new tier of “untouchable” accounts was created, for whom the standard rules and regulations did not apply. In other words, if their content was flagged for violating Facebook terms and conditions, it usually faced no subsequent review. In total, the XCheck database included at least 5.8 million “VIP” users who faced no consequences in case they spread problematic content.

It comes as no secret  that Facebook’s business model operates on the basis of the attention economy. The user’s attention to content can directly be transformed to monetary gains – through ads. 

For a content platform, like Facebook, it is paramount to keep its users’ attention for as long as possible. For that, engaging content for the user must be presented. 

This is handled by Facebook’s recommender algorithms that decide what content is deemed worthy of being seen by the users. 

It is designed to promote lasting and repetitive engagement. Factors such as content popularity, user’s personal preferences, but also emotionality or controversiality of the content and user’s emotional state may enter algorithm’s decision making. As Facebook’s algorithm is quite good at this, it generates revenue in billions of dollars. 

According to Facebook, their algorithm takes into account more than 10 000 signals and evaluates them to provide personalized content. The specific values and importance assigned to these signals are not publicly known, thus rendering the algorithm opaque and nontransparent. 

However, what the Facebook papers made clear is that since 2017, Facebook´s ranking algorithm treated emoji reactions (e.g. anger emojis) as five times more valuable than regular likes. 

This is problematic on many levels, and as we found out based on the internal  communication of Facebook employees, they had known about the potential risks of such a decision. What this means is that controversial content which sparks angry reactions is spread more rapidly. Additionally, the internal documents seem to suggest that Facebook was aware of the fact that such content is more likely to include disinformation, toxicity and low quality news.

Facebook publicly declared that it uses algorithms to detect and moderate problematic content on its platform. However, the leaked documents suggest that its success rate is virtually non-existent. Contrary to Facebook´s public claims that AI results in 98% of proactive detection rate of problematic content, the papers suggest that this number is drastically lower and that the algorithm often struggles to correctly label even extreme content such as videos of violent shootings. In fact, according to the leaked documents the AI systems were removing only 0.6% of all content that violates Facebook policies against violence and incitement. 

There have been mixed reactions to the controversies that Facebook papers have brought up. Some voices claimed that such practices are not surprising at all. Others raised their hopes that the whistleblower Frances Haugen´s accounts have the possibility to influence future developments. 

These hopes have been materialized in Frances Haugen´s testimony in the European Parliament at the beginning of November 2021. In her testimony, she praised the EU lawmakers and their efforts to increase online safety and transparency of algorithms through the Digital Service Act (DSA). She pointed out that this legislation indeed has the potential to address many of the issues that emerged from Facebook papers. Yet, Haugen has also pointed out some problematic aspects of the DSA that could serve as loopholes. As we at KInIT have recently also pointed out, these include the so-called “news media exemption” that would reverse years of progress in the fight against hate speech and disinformation by preventing these platforms from moderating content coming from press publication or the trade secret exemption which might impede the efforts of making the algorithms more transparent. 

There are several key takeaways to be taken from the Facebook papers. It was made rather apparent that Facebook prefers user engagement even if this entails the spreading of problematic content. Besides this, the controversy suggests that Facebook fails to effectively moderate content in most of the world and that many of the public statements made by Facebook are in dire contrast with the actual reality supported by its own internal research data and internal communication among the employees. 

Facebook is one of the platforms where it is very hard for researchers to perform any external audits to check the validity of their public claims on how they fight disinformation and offensive content. For example, it is very hard to reliably measure how often a particular piece of (misinformative) content was recommended, let alone in which social bubbles. From this point of view, KInIT has proposed several times that it will be essential to allow researchers to do continual audits of online platforms such as Facebook or Youtube.

Furthermore, it is hard to establish the validity of some of the leaked data without having access to the detailed analyses, including the research designs and statistical appendices of Facebook´s internal research. It might prove especially beneficial for the researchers and news organizations that were left out, particularly in Slovakia and other post-communistic countries where disinformation resilience is problematic. And as we mentioned earlier, the inaccessibility to data and the intransparency of the algorithm also leads to much wider problems related to the inability to perform external audits, which we believe to be a fundamental tool and measure to increase the accountability of social networks. 

This article was also published in the august issue of the Quark magazine (Slovak version). The article is available here.