In February this year,GoogleAnnounced with Reddit A new agreement was reached.Reddit data will be used to train Google's AI Models, and highlighted Reddit results in Google search results. Bloomberg reported that the deal was for about $60 million. Since then, Reddit's presence in Google searches has also increased significantly.
The latest news says thatReddit has banned other search engines from indexing its content404 Media first reported the move, noting that Reddit had updated its robots.txt File (the protocol file between the website and the crawler) prevents all robots from crawling any content of the website.
Query Reddit's robots.txt The platform said, “Reddit believes in an open internet, but not in the abuse of public content,” the document learned.Its "Disallow" option is set to "/", which means that all files under the root directory of the website are prohibited from being retrieved.
The document was first updated in June of this year, with Reddit explaining that the change was due to "a significant increase in commercial entities scraping Reddit content and the abuse of that content."
Foreign media 9To5Google said,Search engines such as Bing, DuckDuckGo, Mojeek and Qwant were affected, use "site:reddit.com” When searching for content, either the latest content is not displayed or the complete website results are not displayed.
However, the latest testBing and DuckDuckGo have re-displayed “site:reddit.com"Search content.
Reddit spokesperson Tim Rathschmidt said in a statement to The Verge:This is completely unrelated to our recent partnership with Google.We have been in discussions with multiple search engines but have been unable to reach agreement with all of them because some have been unable or unwilling to make enforceable commitments regarding their use of Reddit content, including their use of AI training.”