Reddit is taking a stand against AI companies, or at least asking them to pay up.
Earlier this week, Reddit announced that it was working on changing its robot exclusion protocol, also known as robots.txt. This seemingly dry tweak is part of a larger negotiation/battle between AI companies hungry for content they can use to train their language models, and companies that already own the content.
“Robots.txt” is the way websites communicate with third parties about how to crawl a website – a classic example is websites that allow Google to crawl them so they can be included in search results.
In the case of AI, the value exchange is much less clear. When you run a website whose business model is to attract clicks and eyeballs, there’s little appeal in letting AI companies animate your content and then send you no traffic — and in some cases, steal your business entirely.
So, by changing its robots.txt file, and also by continuing to cap the rating and ban unknown bots and crawlers, Reddit appears to be working to prevent practices that companies love. Artificial intelligence has been criticized for being confusing..
Press play to learn more, then let us know what you think in the comments!