From Search Rankings to AI Agents: The New Rules of Visibility

Ben cohen

Here's a scenario. An operations manager needs a new scheduling tool for her team. She opens an AI assistant, types a long, messy description of what she needs, and gets back a shortlist of three tools, a comparison of how they differ, and a recommendation. She picks one. She may not visit a single website until she's ready to buy.

Ido

And if your product wasn't one of the three, you don't get a bounce rate. You don't get a lost session in analytics. Nothing. You just quietly weren't in the room.

Ben cohen

That's the part that bothers me. In SEO, at least you knew when you were losing. You could see the ranking drop. Here the loss can be completely invisible.

Ido

So that's today's question. How does a brand stay visible when the thing standing between the customer and the web is an AI assistant?

Ben cohen

Welcome to Be Visible. I'm Ben Cohen.

Ido

And I'm Ido Navarro. This show is about AI visibility, SEO, answer engines, and getting websites ready for agents. Today we're going from the big shift, to the website plumbing, to agents that act on your site, and then a checklist you can start this week.

Ben cohen

And we'll try to be honest about what works today, what's still experimental, and what's a hypothesis wearing a confident outfit. There's a lot of that in this space right now.

Ido

A lot. So, what actually changed? Traditional search gave you a page of links. Your goal was to be high on that page, and position was the thing you tracked. Generative engines, think ChatGPT, Claude, Perplexity, Gemini, Google's AI Overviews, give you a written answer. And that answer is generated, which means it's probabilistic. Ask the same question twice and you can get slightly different answers, with different brands mentioned.

Ben cohen

Which breaks the idea of a fixed position. I watched a video recently reviewing AI visibility tracking tools, and one line stuck with me. The speaker said, and I'm quoting, Rank isn't really something for us to look at. We're really looking more at like brand mentions and citations.

Ido

Right. If the answer changes from run to run, position one isn't a stable thing you can own. So the metrics change. How often does your brand show up across a set of prompts? Is it cited? Is the tone positive? Is what the model says even correct?

Ben cohen

Correct is the one I'd underline. Visibility used to mean being found. Now it's a chain. You have to be discovered, understood correctly, considered credible, quoted, and then recommended. If any link breaks, you're out. Or worse, you're in, but the assistant says you don't offer something you do offer.

Ido

Or gets your pricing wrong. That's a real concern with product data, and it's one reason some tools in this space talk about syncing product and price information.

Ben cohen

Okay, let me play the skeptic, because someone listening is thinking it. Is this really different from SEO? Or are we renaming SEO and selling it back to people?

Ido

Partly, no, it's not different. The foundations carry over. Your site has to be crawlable, reasonably fast, useful, and trustworthy. The GEO Lab WordPress guide makes a point I like. Many site owners have done the SEO basics, keywords, meta descriptions, sitemaps, but haven't made the leap to optimizing for AI answers. So GEO is more like new layers on top than a replacement.

Ben cohen

What are the layers?

Ido

First, extractability. Can a machine lift a clean answer out of your page? Picture two pages. One buries the answer in the fourth paragraph under a hero animation. The other has a question as a heading and a direct two to three sentence answer right under it. The second one is much easier to quote.

Ben cohen

Second layer, for me, is what happens off your site. If the only voice saying you're great is your own homepage, a model has one source. Brand mentions across trusted places probably shape how AI understands you. And I'll flag that as a reasonable belief, not a proven formula. Nobody outside the labs knows how much weight any of it carries.

Ido

Good flag. Entity clarity is the third layer. Who are you, what do you do, who do you serve, who wrote this, why should anyone trust it? Humans infer that from design and context. Machines need it spelled out.

Ben cohen

So are we optimizing for the model, the crawler, or the user?

Ido

The user is still the point. The crawler and the model are the routes the user's question travels through. If you make the site worse for people to please a machine, you've lost the plot.

Ben cohen

Let's get into the plumbing then. I built a free tool, the AI Content Reader at GetAISO, that strips away styling, JavaScript, and visual layers to show what a site looks like to systems like ChatGPT, Claude, Perplexity, and Gemini. The whole idea is that what you see in a browser and what a machine reads can be two very different things.

Ido

Here's the analogy I'd use. A website is a beautifully designed restaurant. The lighting, the furniture, the music, all of it matters to a guest. But an AI doesn't need to inspect the furniture. It needs the menu. Clear, written, here's what we serve and what it costs. Clean markdown is that menu.

Ben cohen

And why does the noise matter? Machines can read HTML.

Ido

They can, but models work within limits. There's a context window and a processing budget, measured in tokens. Modern sites, especially ones built with heavy visual page builders, can carry a lot of nested containers, tracking scripts, repeated navigation, and styling code. Some people call it a token tax. The model spends effort wading through markup instead of reading your actual claim.

Ben cohen

The GEO Lab guide goes as far as listing Elementor, Divi, and WPBakery under plugins to avoid for blog content, because of code bloat.

Ido

And I'll be a little defensive, since I've spent years building with those tools. Their advice is to use builders for landing pages and avoid them for articles, which is a fair nuance. And honestly, the fix isn't always to abandon the builder. It's to make sure the content layer underneath is clean. That's the part to test, not assume.

Ben cohen

What's the evidence that cleaner pages actually get cited more?

Ido

Honestly, it's thinner than the confidence in the marketing suggests. A tool called Salespeak generates streamlined versions of pages for AI bots, stripping out template HTML and extra containers. In that same tool review, the speaker said that if you limit the amount of work these tools have to do, that can be a positive signal for retrieval. Notice the wording, can be. It's logical, and the direction makes sense, but I'd call it a well reasoned hypothesis, not a guaranteed ranking factor.

Ben cohen

And it points to something bigger. Eventually the same content may be served in different forms. A rich page for humans, a clean text version for machines, maybe a structured action layer for agents.

Ido

With one big rule. It has to say the same thing. If the machine version makes claims the human version doesn't, you're not being helpful, you're being deceptive, and that can backfire.

Ben cohen

Okay, the question everyone asks. llms.txt. Does adding one suddenly make my site visible?

Ido

No. Let me explain what it is and then what it isn't. It was proposed in September 2024 by Jeremy Howard of Answer.AI. It's a markdown file at the root of your site, a curated, low noise index of your most important pages, with short descriptions. The companion idea, llms full dot txt, goes further and puts the complete content of those pages into one file.

Ben cohen

So one is the table of contents, and the other is the whole book.

Ido

Exactly. Mintlify adopted it in November 2024, and some developer focused platforms, Anthropic and Cursor among them, use it, mostly around documentation. That's where it's genuinely useful today. A developer's coding assistant can pull a clean version of the docs.

Ben cohen

But for a normal business website?

Ido

Here's the reality check. Public chat assistants don't systematically crawl llms.txt as an automated ranking signal for everyday commercial sites. As far as we can tell, that isn't how it works today. So treat it as experimental and cheap. Its biggest benefit right now might be internal. Writing it forces you to decide which ten pages matter most and how to describe them in plain language.

Ben cohen

That alone is worth an afternoon. And you can write it by hand. It's a text file. Some tools generate and maintain it for you, LLMagnet has a generator for that, and the appeal is keeping it updated as the site changes. But it's one piece, and a hand written version beats an unchecked generated one.

Ido

Agreed. Now schema, which is the more established piece. The GEO Lab guide calls schema the ID badge your content wears for AI. Structured data tells a machine explicitly, this is an article, by this person, modified on this date, from this organization.

Ben cohen

Which types matter?

Ido

Organization, so who you are. Article, with dateModified, not just the publish date, because it signals the content is maintained. Author or Person, linked to real profiles and credentials. Then whatever fits your business, Product, Service, FAQ, Review, Event, LocalBusiness. The FAQ type is singled out by that guide as one of the most directly extractable formats for AI, since you're literally pairing questions with short answers.

Ben cohen

I'd add a caution. That guide also says FAQ pages are significantly more likely to be cited. That's their claim, and I haven't seen the data behind it. Still, clean question and answer pairs are good for humans anyway.

Ido

Same logic. And the mistakes are where people get hurt. Schema that claims things the visible page doesn't say. Duplicate schema because someone runs Yoast and Rank Math together. Generic markup stamped on every page. And the classic WordPress one, posts still attributed to admin or webmaster. The guide treats that as a trust red flag, and it makes sense. A nameless author is a weak entity.

Ben cohen

And schema can't rescue weak content. Labels on an empty box are still an empty box.

Ido

Perfect. One more basic thing, and it's the most embarrassing one. Crawler access. Check your robots.txt. Make sure you aren't blocking GPTBot, Google Extended, ClaudeBot, or PerplexityBot by accident. The GEO Lab notes that some WordPress security plugins and firewalls block AI bots by default.

Ben cohen

To be fair, blocking can be a legitimate choice. A publisher protecting its content from model training might do it on purpose. But then you have to accept the trade. If the systems can't read you, you're probably not in the answers. Just make it a decision, not an accident. And check what each bot is actually for before you block or allow it.