The tech giant now displays AI-generated answers – called AI summaries – at the top of its search pages, often preventing users from clicking through to the sites that originally provided the information.
This puts publishers in a dilemma: They must either allow Google to use their data for these AI summaries or risk losing their visibility in search results entirely.
The problem is that the Google tool responsible for sifting through web content to generate AI answers is the same one the company uses to index web pages for search results.
Publishers say they cannot block Google’s crawler (Googlebot) because it also indexes websites for search rankings. Blocking it would make their sites almost invisible on the web.
“It’s becoming an existential crisis for these companies,” said Joe Ragazzo, editor of the news site Talking Points Memo, Bloomberg.
“Those are two bad options. Either you quit and die immediately, or you partner with them and probably die slowly because at some point they won’t need you anymore either.”
Google’s dominant position in search, which a federal court recently ruled was an illegal monopoly, has made the company a powerful player in the emerging AI industry.
This has led to accusations of unfair competition, especially as the AI search landscape takes shape. Publishers in particular are faced with the difficult decision of contributing to AI models that may eventually render their sites obsolete, or losing a key traffic source.
Google, for its part, has tried to allay these fears, saying its search engine continues to drive billions of visits to websites, creating value for publishers.
“Every day, Google sends billions of clicks to sites across the internet, and we intend to continue this long-established exchange of value with sites,” a Google spokesperson said.
“With AI Overviews, people find search more useful and return for more frequent searches, creating new opportunities for content discovery.”
In May, Liz Reid, Google’s vice president of search, acknowledged that there were concerns before the launch of AI Overviews, but suggested there were positive signs during testing.
“User feedback shows that with AI Overviews, people are happier with their search results and are asking longer, more complex questions that they know Google can now help them with,” she said.
“They use AI Overviews as a starting point to visit web content, and we see that clicks on web pages are higher quality – people are more likely to stay on that page because we have better found the right information and helpful web pages for them.”
In recent years, the rise of generative AI has given rise to a new wave of startups seeking to offer AI-powered search products. But these startups face a major hurdle: building a comprehensive index of the web.
Crawling websites, i.e. visiting and indexing web pages, is resource-intensive and expensive, making it challenging for new market entrants to compete with established providers such as Google and Microsoft Bing.
Alex Rosenberg, CEO of AI startup Tako Inc., explained the challenges facing smaller companies, noting that it is difficult for search startups to guarantee traffic before they gain a foothold.
This is one of the reasons why these young companies have started to enter into agreements to pay publishers licenses for content.
“Today there are a lot of technology companies that are paying for content, they are paying for access to it because they need it to be able to seriously compete,” Rosenberg said.
“Google, on the other hand, doesn’t actually have to do that.”
As the legal battle over Google’s monopoly continues, the Justice Department is reportedly considering remedies that could force Google to share its search data with competitors or even break up the company.
Regardless of the outcome, changes in the search landscape mean publishers must diversify their traffic sources and reduce their reliance on a single technology platform.
