Optimizing for AI: The Death of Rank and Rise of Agents

Ben cohen

Okay, imagine this. Someone runs a twelve person agency and needs a new project management tool. They open an AI assistant, describe their team, their budget, and the one thing they hated about their last tool. They get a shortlist of three products, a short comparison, and a recommendation. Then they go sign up for one of them. They never visited the websites of the other two. Those companies never saw a pageview, a bounce, or a lead form. They just lost.

Ido

And what gets me is that the loss wasn't about design or pricing. It might have been that the assistant couldn't tell what the product does. Or it understood it slightly wrong. Or it understood it fine and had no reason to trust it enough to say its name.

Ben cohen

So that's the question for today. How does a brand stay visible when the interface between the customer and the web is increasingly an AI assistant?

Ido

Welcome to Be Visible. I'm Ido Navarro. I spend my days on website building, WordPress, and lately agentic experiences, where you tell an AI what you want and it builds or manages the site.

Ben cohen

And I'm Ben Cohen. I come at this from the growth side. How do customers find you, and can you turn that visibility into actual business. I've also been obsessed with seeing what a website looks like to an AI once you strip away all the styling. We'll get to that.

Ido

Last time we talked about how buying decisions increasingly happen inside AI shortlists before a human ever lands on a site. We touched on robots.txt and code weight. Today we want to go wider. Who is a website actually for now? And what does it need to do for each of them?

Ben cohen

Let's start with what changed, because I think people hear AI search and just assume it's Google with a chatbot on top.

Ido

It's a real shift, though. Traditional SEO was built for an indexer and ten blue links. You used keywords, metadata, backlinks, and the goal was a position on a page. The result was static enough that you could say, we rank number three for this term. Now you have models like ChatGPT, Claude, Gemini, and Perplexity generating an answer. And that answer is probabilistic. Ask the same question twice and you might get a slightly different answer, with different brands mentioned.

Ben cohen

Which breaks the idea of a ranking position. If the answer changes every time, what are you even tracking?

Ido

Right, and there's a line I liked from a video reviewing AI visibility tools. The speaker said, quote, Rank isn't really something for us to look at. We're really looking more at like brand mentions and citations. That's one marketer's framing, but it matches where the tooling is going. You see metrics like visibility score, how often your brand appears, citation share of voice, average position inside an answer, sentiment, and what they call organic overlap, meaning how much your traditional rankings line up with what the AI actually cites.

Ben cohen

So visibility used to mean being found. Now it's more like a series of gates. Can the system discover you? Does it understand you correctly? Does it consider you credible? Does it cite you? Does it recommend you at the moment someone decides? And then, can an agent actually do something on your site?

Ido

Yes. And you can fail at any gate. You can be discovered and misunderstood. You can be understood and not trusted.

Ben cohen

But here's the question a lot of listeners are asking right now. Is this really different from SEO? Or is it SEO with a new name so consultants can sell new packages?

Ido

Fair challenge. Honestly, a lot carries over. Fast site, crawlable pages, clear structure, good content, real authority. If your SEO foundation is a mess, GEO won't save you. I don't see them as enemies. GEO sits on top.

Ben cohen

So what's the new layer?

Ido

I'd say three things. First, extractability. Can a model pull a clean answer from your page, summarize it, compare it with another product? Direct answers, clear definitions, and original data help there. Second, entity clarity. Does the system know who you are, what you do, who you serve, and who wrote the content? Third, corroboration. What do other trusted sources say about you? A model forms a picture from more than your own website.

Ben cohen

That third one matters for business. If everything good about your brand is only said by your brand, that's a weak signal. Mentions on trusted third party sites, reviews, articles, communities, those shape the picture. Though I want to be honest, nobody outside the labs knows exactly how much weight each system gives to those mentions. That's an informed hypothesis, not a proven formula.

Ido

Agreed. I'd label it likely important, not precisely measured. Okay, here's the framework I keep coming back to. A modern website has three audiences. The human visitor, the machine that interprets the information, and the agent that wants to take an action.

Ben cohen

Walk me through each, because I think the human one is where everyone already lives.

Ido

Humans want visual design, storytelling, emotion, an interface that feels intuitive. That doesn't go away. Machine interpreters, the LLMs, answer engines, and AI search products, want structured, low noise facts. Clear definitions, clear entities, proof they can extract. And agents, shopping bots, research assistants, comparison tools, website management tools, they need predictable navigation, labeled form fields, unambiguous pricing, and some way to complete a task.

Ben cohen

Here's the analogy I'd use. Your website is a beautifully designed restaurant. Great lighting, a lovely room, the right music. A human walks in and feels something.

Ido

And the machine just wants the menu. What's on offer, what's in it, what it costs. It shouldn't have to inspect the furniture, the wallpaper, and the light fixtures to figure out whether you sell pasta.

Ben cohen

That's actually what my AI Content Reader shows people. It strips away the styling and the JavaScript, and shows the page the way an AI system might see it. People are often shocked. Their gorgeous homepage turns into a pile of repeated menu items, cookie text, and vague headlines. Meanwhile the one sentence that says what the company does is missing.

Ido

I have to own something here. I've built hundreds of sites with visual builders, so I'm biased toward them. But a guide from The GEO Lab on WordPress makes a pointed claim, that heavy page builders like Elementor, Divi, and WPBakery can generate a lot of code bloat that's hard to parse cleanly, so it suggests using them for landing pages and not for articles. I'd soften it a little. Builders aren't the villain. It's the nested wrappers, the repeated layout chrome, the scripts. That's what makes the raw HTML heavy.

Ben cohen

And why does heavy HTML matter? If the AI is smart enough, can't it just ignore the clutter?

Ido

Often it can. But think of it as a token tax. Context windows and parser budgets are finite. The more markup stands between the system and your actual content, the more work extraction takes, and the more chances something important gets dropped. One tool reviewer in that video said that if you can limit the work these tools have to do, that can be a positive signal for retrieval. I'd call that plausible, and a claim to test, not a guarantee.

Ben cohen

So that's where clean markdown comes in.

Ido

Right. Markdown is a plain text version of the page. Headings, lists, links, tables. No navigation, no scripts, no repeated interface elements. It's the menu. And I think we'll see sites serving different representations of the same content. The rich page for humans, a clean version for machines, and eventually something structured for agents. Some tools already do a version of this. Salespeak, for example, was described as generating streamlined, AI optimized versions of pages by stripping template code.

Ben cohen

Okay, now the file everyone asks about. llms dot txt. Does adding one suddenly make a website visible?

Ido

No. Flat out no. Let me say what it is. It's an emerging convention, a concise markdown file at the root of your site that works like an index. Here's who we are, here are the most important pages, here's what each one covers. Its bigger sibling, llms full dot txt, goes further and includes the actual content in a clean structured form, so a system doesn't have to crawl page by page.

Ben cohen

So the first is a table of contents, and the second is the whole book.

Ido

Pretty much. And the work in both is curation. Which pages matter? How do you describe them? What do you leave out? That's a content strategy decision. But here's the honest part. It's still an emerging convention. I wouldn't tell anyone that every major AI system reads it, or that it changes rankings. Treat it as a cheap, sensible aid and one part of an AI readiness strategy. If your actual content is vague, a nicely formatted index of vague pages is still vague.

Ben cohen

I like that, because business owners want one magic lever. Next is schema, which sounds technical but I think people can grasp it.

Ido

Schema is structured data that gives machines explicit labels. This is an article, by this author, about this topic, published and modified on these dates. This is an organization, with this logo and these profiles. This is a product, with this price. The GEO Lab guide calls it the ID badge your content wears. And it lists the useful types as Article, Organization, Person for authors, and FAQ, plus others like product, service, review, event, and local business depending on what you do.

Ben cohen

One thing from that guide that jumped out was the admin problem. Posts published under admin or webmaster.

Ido

Yes, the guide treats that as a trust red flag for AI. I'd be careful with how strongly we state it, because we can't see inside these systems. But as a common sense matter, a real author with a bio, credentials, and consistent profiles across the web is a better signal than a username. It's also just better for humans. Who wrote this and why should I believe them?

Ben cohen

And the date thing. dateModified, not just datePublished.

Ido

Right, it's a freshness signal. It says someone maintains this. Only, and this is important, it has to be true. Which brings us to schema drift.

Ben cohen

Drift, meaning the structured data and the visible page stop matching.

Ido

Exactly. The JSON LD says the product is in stock and the page says sold out. The schema claims a four point eight rating that the visible page never shows. Or someone updates the price on the page and the plugin keeps the old one. Now the machine gets two stories, and trust goes down. Schema should match what a human can see. And it can't rescue weak content. It labels what's there.

Ben cohen

The same guide calls FAQ schema one of the most directly extractable formats, because it's pre packaged questions and answers. It claims pages with it are significantly more likely to be cited. I'd treat that as a claim from one guide, not settled fact. But it makes sense. If the real questions customers ask have clear two or three sentence answers, that's useful to a person and to a model.

Ido

One more infrastructure piece, and it's the most boring and the most dangerous. robots.txt.

Ben cohen

The accidental invisibility problem.

Ido

Yes. Some security plugins or hosting rules block unfamiliar bots by default. GPTBot, ClaudeBot, PerplexityBot, Google Extended. And the owner has no idea. There's a legitimate choice there. Maybe you want to protect proprietary content from training. But that should be a decision, not an accident. Go to yoursite dot com slash robots dot txt and just read it.