Content moderation

From Tournesol
Revision as of 16:11, 26 March 2021 by imported>Aidjango (page creation)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Content moderation has became a huge task for social media platform such as Youtube or Facebook. As these medias have a very large audience, they have to deal with a lot of malicious behaviors for marketing, misinformation, radicalization but also political purpose. The amount of data uploaded everyday on theses platform make the moderation impossible to be entirely performs by humans. For example, everyday:

  • 82 years of videos are uploaded on Youtube TF-19
  • 16,6 millions of fake account are deleted from Facebook NYT-20, 99% of them before it has been flag by a user.
  • 86 million of picture are uploaded on Instagram OMNI-21

A large part of these content are created farm bot as shown in the series of videos by SmarterEveryDay about social media manipulation SMM-pl.


The content moderation is perform through three main technique: algorithmic moderation, human moderator and user reporting content. In practice, the interactions between users, flags, algorithms, content moderators, and platforms are very complex.

Algorithmic moderation

Machine vision AI is fooled by handwritten notes. Image from VER-21 based on the work from OpenAI-21.

With improvement in image recognition, automatic speech to text and fraud detection, a large part of the moderation can be performed automatically. However, there is still a lot of cases where algorithms are unable to differentiate honest and malicious content or can be fooled by simple tricks OpenAI-21

Human moderator

For example Facebook had around 20'000 human moderators in 2018 FOR-18. Some of these human moderators suffer from trauma symptoms VER-19 due to the daily exposition to hate speeches, videos of violent attacks and the graphic pornography.

User reporting content

Users of the platform can also signal ("flag") content they found inappropriate. It can also lead to some mechanism of governance to avoid the diffusion of certain type of information. CrawfordG-14