(AI) bots eat 30% of the web bandwidth

 One of the hidden costs of AI is the hunt for training data by ripping content from websites. 

The bots are massively using content from websites without paying for it, ignoring any copyright that may be associated to that content. They are even using AI to find content that is hidden behind JavaScript logic.

AI providers often argue that they refer to their sources and that this leads to more visists to those websites. Cloudflare, a major CDN provider, has shown that often the referrals are very low compared to the bot traffic, OpenAI being the worst with 1 referral for 1600 crawls:

 

Cloudflare is now preparing a service to install a payment wall for crawling. 

Not only are the AI bots not paying for the content they are sucking from the content providers. Again according to Cloadflare, they represent 30% of the traffic going to these websites. And who is paying for the consumed bandwidth?Indeed, the content providers themselves are paying for the bandwidth their servers need.