Shelly Palmer was named a “Top Voice in Technology” by LinkedIn and writes a popular daily business blog.
For those of you who are not yet of a certain age, at the height of the original Ghost Hunters In the film, Gozer the Gozerian (an ancient Sumerian demigod, also known as Google) offers the Ghostbusters a choice (albeit not a great one): they get to choose the form of the monster that will destroy humanity. As is often the case in Hollywood movies, it worked out quite well; evil marshmallow monsters can end up roasted. Even if Google isn’t a giant Stay-Puft Marshmallow Man, the choice it offers publishers is absolutely existential.
To provide search results pages and AI summaries, Google crawls the web with its Googlebot. The company says AI summaries are part of its commitment to providing higher quality information and creating opportunities for publishers. But publishers face a dilemma: They can’t stop Google’s AI from summarizing their content without also hurting their site’s discoverability in search results, because the same Googlebot serves both purposes.
While Google allows publishers to exclude certain pages (or parts of pages) from appearing in AI summaries, doing so would likely result in those snippets not appearing in all of Google’s search features, including web link listings. This puts publishers in an impossible position, as many of them rely heavily on search engines for their traffic.
As you can imagine, the situation is a cause for concern for publishers. Alphabet and Google are often wrongly accused of monopolistic behavior, but this is not the case. We need a solution to Google’s reinvented publisher dilemma. Hey Google… Can you offer us an alternative to destruction by deindexing or destruction by scraping? I’m thinking of compensation for content creators and publishers, but I’ll leave it to you to come up with a better offer.
As always, your thoughts and comments are welcome and encouraged. Just reply to this email. -s
PS: If you want to better understand how AI bots crawl the internet for model training, check out our free online course Generative AI for Brand Marketers to help you start your AI journey.
ABOUT SHELLY PALMER
Shelly Palmer is Professor of Advanced Media in Residence at the SI Newhouse School of Public Communications at Syracuse University and CEO of the Palmer Group, a consulting firm that serves Fortune 500 companies in technology, media and marketing. He was named a “Top Voice in Technology” by LinkedIn and covers technology and business for Hello, New Yorkis a regular commentator on CNN and writes a popular daily business blog. He is a best-selling author and the creator of the popular, free online course Generative AI for managers. Consequences @shellypalmer or visit shellypalmer.com.
