Whistleblowers at Meta's Integrity Organization have shared data with International Corruption Watch (ICW), revealing evidence of a mass censorship strategy abusing Meta's reporting system.

Normal Takedown Process

User Reports—Any Facebook user can flag a post.

AI Screening—The post is first checked by a content enforcement AI model that reviews the text and associated media. If the model is confident, it will remove the post.

Human Review—If the model is not confident, the report will be escalated to a human reviewer.

Training Loop—If a human approves the takedown, the post is labeled as a piece of training data and fed back to the AI's training dataset, allowing for the model to adapt in real-time.

Meta's public description of this pipeline is available at https://transparency.meta.com/enforcement/detecting-violations/how-enforcement-technology-works.

Prioritizing Government Requests

Governments and privileged entities have special access to submit takedown requests. These requests are given priority and are sent directly to human reviewers. They can be submitted by form or direct email to Meta depending on which country is involved.

These reports are handled faster and are more likely to be removed. Similarly, when approved by a human, it is given a label and added to the training dataset.

When large amounts of government-submitted requests are approved, the AI model receives a flood of labeled examples that it further trains itself on.

When abused, this is an example of a machine-learning attack called a "data-poisoning" attack. The model eventually becomes heavily biased towards removing content that matches the pattern of reports.

International Corruption Watch Investigation

The ICW theorizes that governments like Israel use mass reporting during a crisis to shape public opinion. In this report, ICW aggregated data and then analyzed takedown requests from the Israeli government.