Should Publishers Block AI Bots? Keep Google, Block the Rest

10% of people get news from AI chatbots every week. Among 18- 24-year-olds, it’s 17%, and both numbers keep climbing.

That’s from the Reuters Institute’s Digital News Report. Google’s AI Overviews pull from your articles, answer the question and keep the reader on Google.

So should publishers block AI bots? A Tow-Knight Center report says blocking hurts the public. But let AI in and it answers your readers’ questions for you, so they never reach your site.

The answer for small and mid-size publishers: block selectively, keep Google, and turn the website visits you still get into email addresses before those visits dry up.

Listen on the Paywall Podcast

On this episode of the Paywall Podcast, Pete works through the Tow-Knight Center’s report, The Three-Layer Cake, and answers the question publishers keep asking him: if you let AI in, how do you keep your readers? His advice comes from the registration walls and paywalls he runs for publishers on WordPress.

Why Publishers Shouldn’t Block Every AI Bot

The Tow-Knight report, by Gina Chua and Adiel Kaplan, says a publisher who opts out of AI gives up on informing most of the public and leaves that job to others. Individual publishers might survive, they argue, but the wider pool of public information gets poorer.

Block AI and your reporting disappears from where more and more people now get their news. Let it in and you lose the visit, and within a year or two, website visits might be a trickle.

Stay in Search and Still Own Your Readers

The Tow-Knight report breaks it into three layers: your content, the AI bots and agents that pull it, and the reader at the end. You can deal with the middle layer and still reach the reader directly.

Let Google’s bot in and shut out the rest, so search still sends you readers. Then use the window you still have to turn website visits into email addresses.

Looking to grow your publication?

Sign up for expert advice straight to your inbox.
This field is for validation purposes and should be left unchanged.

What to Do Today

How much AI you block depends on your size. An established brand with a big audience can block every AI bot. A typical mid-size publisher that still relies on that traffic should be more selective. Licensing your content to AI is the other option: the tools make it easy to set up, but for small and mid-size publishers the revenue is pennies, so it can wait.

Publish what AI can’t copy. If readers can get it somewhere else, stop. Focus on original reporting, your opinion, your judgment, your voice, podcasts and events.

Block every AI bot except Google’s. Google Search and AI Overviews run on the same bot, so blocking it costs you search traffic.

Put a registration wall on article 2. Give away one free article. On the second, ask for an email address in exchange for a few more free articles a month (one to three works). The registration wall is probably the most effective tactic in the industry.

Match the wall to your publication. News sites can wall everything, which also hard-locks content from bots. You lose some search and social traffic, but the traffic you keep converts well into email signups. Magazine and niche publishers can lock premium features behind a paywall and keep a registration wall on news, so news still pulls search and social traffic while it builds the list.

Your newsletter then becomes one of your most important products. Send it often, treat free and paid readers well, and it brings traffic back to your site. We call that the Publisher Flywheel™.

Mexico News Daily runs a registration wall on article 2, a paywall on article 3 and two daily newsletters. Traffic grew from 500,000 to over 1.4 million monthly visitors in about a year and a half.

Colin Devroe, head of AI at NerdPress, told Pete that in a few years AI agents will be your readers and people won’t go to websites. He might be right, and down the road you’ll likely sell access to your readers’ AI agents too. Leaky Paywall is already building for that. But agent traffic is still low, so the window today is the email address. Every reader you register now is one you can reach when visits slow down.

Key Takeaways

  • Keep Google’s bot and block the rest. Search and AI Overviews share one bot, so blocking it costs you search traffic.
  • Trade content for email: one free article, then a registration wall, while website visits still come in.
  • News sites can wall everything. Magazines and niche publishers should hard-lock premium features and keep news open to search behind a registration wall.

What to Try This Week

  1. Check which bots you block today. Make sure Google’s bot isn’t one of them.
  2. Set your free article limit to one. Then turn on a registration wall with List Builder.
  3. If you run a magazine or niche site, list your top feature articles. Mark each one as paid-only or behind a registration wall, then lock the paid-only ones.

Watch the full conversation:

Learn how Leaky Paywall can help grow your subscription revenue