In the first quarter of 2025, increasing AI traffic from advanced bots unleashed onto thousands of websites by companies like OpenAI began transforming information by saving and summarizing content in real time and upsetting traditional web traffic on news sites.
The new AI-based bots generation indicates more than mere web crawl technology development. By searching hundreds more pages than a human would, these bots provide AI programs that give fast, easy-to-read answers instead of just links.
These bots’ behaviors are already impacting sites globally, causing publishers to rethink their strategies on visitors’ treatment, and the bots’ growing frequency exposes a radical change in digital engagement, and an evolution driven by enhanced AI traffic and fluidity.
AI Retrieval Bots
Many are dropping Google searches and replacing them with AI features like ChatGPT. With this transition came a new kind of web crawler that both reads and real-time web content summaries.
Instead of displaying the old-fashioned list of links, AI now gives clear and quick answers.
OpenAI and Anthropic are a perfect example of how this deployment of bots streamlines the gathering of relevant web-data, by uploading additional content into AI’s memory – which can hold way more than any human would be able to read.
AI models such as these provide immediate, on-the-spot summaries, changing mannerism of users’ engagement with online content, highlighting how AI for traffic management and content optimizes dense data flows through internet.
Statistics shared exclusively with The Washington Post from TollBit – startup that helps news publishers track AI consumption of their news – show AI website traffic from automated sources rose 49% in 2025’s first quarter (Q1), compared with 2024’s last quarter.
TollBit analyzed 266 sites, including most major news outlets, and concluded that the number of AI bots demanding data on demand is growing at a fast rate.
“It starts with publishers, but this is coming for everyone,” TollBit co-founder and CEO, Toshit Panigrahi said.
Panigrahi proceeded to say that although fewer people are visiting sites directly, the total amount of AI traffic tools is booming.
“Human eyeballs to your site decreased. But the net amount of content access, we believe, fundamentally is going to explode,” Panigrahi added.
In a defensive effort, OpenAI clarified that while ChatGPT web crawler traffic might be less in number, they are from users with higher target than typical web browsers.
Hurdles Ahead for Sites
Taking all this into consideration, websites will now have to rethink how they serve these AI visitors, instead of just focusing on humans. Panigrahi warned it would be difficult to convince AI companies to pay for web scraper bot content, especially as they make their bots much harder to spot.
Panigrahi highlighted that the impact of AI traffic might be larger as some bots, much like those used by AI agents performing tasks, like ordering food, aren’t fully counted. Chief operating officer (COO) for TIME, Mark Howard, added that most AI bots do not currently pay for content use, meaning web traffic control using AI still faces major challenges. This technology is reshaping the internet.
Websites that are built for Google’s search model must adapt to the rise of AI-driven answers or risk falling behind.
As Panigrahi reassured, optimizing only for human search traffic misses out on the growing potential of AI to increase website traffic.
Inside Telecom provides you with an extensive list of content covering all aspects of the tech industry. Keep an eye on our Tech sections to stay informed and up-to-date with our daily articles.