Meta has deleted a clause from its commercial terms that critics read as permission to scrape news sites for AI training. The removed sentence said Meta could use content from domains, web pages, or URLs to develop, improve, and train its products.
Neiman Lab and Cornell law professor James Grimmelmann both flagged the wording. They argued it handed Meta a free license to scraped material and could override a site’s robots.txt instructions. Publishers reacted fast, because their archives are the raw material for model training.
Meta denied any change in behavior. Vice president of communications Andy Stone and spokesperson Daniel Roberts said crawler practices stayed the same. “Any suggestion to the contrary is false,” they said in a joint response.
The company went further on Saturday. Roberts told The Verge that the consolidated crawling terms only covered ads and business products. “We’ve heard you and want to clarify,” he said. Meta now states that content owners can still use robots.txt to signal their preferences, and that the update grants no new rights that supersede them.
The revision does not settle the larger fight. News publishers have spent years watching AI companies pull their archives into training sets. Some have signed licensing deals, while others have sued or blocked crawlers outright. Every major model release restarts the argument about what a crawler may take.
Robots.txt is a voluntary signal, not a law. A site can ask crawlers to stay out, and a company can choose to honor that request or ignore it. The deleted Meta clause looked like an attempt to make the request optional for one specific use. Removing it puts the old expectation back on paper, even if nothing changes behind the scenes.
For readers, the practical effect is small today. For sites that rely on search and social traffic, the wording matters a great deal. A single sentence in a commercial contract can decide whether years of reporting feed someone else’s model for free.
Meta’s edit shows that public pressure can move contract language even when a company insists nothing changed. The next test is enforcement. Crawler logs will show whether the new terms match the old behavior, and publishers are watching those logs closely.
Read the original report from The Verge. For more on how sites are pushing back against automation, see our report on websites blocking AI agents.
The Netherlands, for internationals
One email a week: the week's news for internationals in the Netherlands, new practical guides and what changed in the cities. No spam, unsubscribe any time.
