Be Visible
All Episodes
Clean Code, Clear Answers: Technical AI Visibility

Clean Code, Clear Answers: Technical AI Visibility

0:00|0:00

Ido Navarro and Ben Cohen explore the technical infrastructure of AI visibility, breaking down how models crawl raw HTML, the role of clean markdown and llms.txt, and why sites must be built for both search engines and autonomous agents.

This show was created with Jellypod, the AI Podcast Studio. Create your own podcast with Jellypod today.


Chapter 1

Why AI Can't See Your Website

Ido

Ahrefs looked at 66.6 million ChatGPT responses and about 15 million fan-out queries. Only 14.4% of those responses triggered fan-out searches at all, and they averaged roughly 1.6 sub-queries. So your page isn't competing in some giant arena. It has to match one or two searches. Miss them, and the model never sees you.

Ben cohen

That connects to our earlier episode on GEO versus SEO, where we talked about fan-outs and third-party mentions. This time I want the technical layer. Because here's what scares me. A brand can win the strategy and still lose, because its prices sit inside JavaScript the crawler never runs.

Ido

Yes. Googlebot crawls the HTML, then does a second pass to render JavaScript. The reporting we've seen says the AI tools tested, including Perplexity Sonar Pro, Gemini 2.5 Flash, Claude 4.0 Sonnet and OpenAI models, load only the initial HTML. So pricing in a client-side widget or behind an accordion that needs a click is effectively invisible.

Ben cohen

And that matters more because of what ChatGPT does when it searches. In the Ahrefs data, about 6.8% of fan-outs used the site: operator, and 75.9% of the most common ones pointed at a brand's own site. The word official showed up in 13.1% of English fan-outs in September. Price fan-outs rose 47% in a month.

Ido

So the model is literally going to your front door and asking for the price. Then it finds an empty shell.

Ben cohen

But does it always trust the brand? Comparisons look different, right? For factual questions, 63.4% of citations went to official sites. For A versus B questions, that fell to 46.4%.

Ido

So on facts you must be the source. On comparisons, others tell your story. Own the facts, earn the mentions. One caveat: Ahrefs also reported Reddit citations down 88% in six months, so those source patterns move fast. Don't build a strategy on one snapshot.

Ben cohen

Okay, so what does clean delivery look like? You've got an analogy.

Ido

Your website is a beautiful restaurant. Lighting, furniture, music. Great for guests. A clean markdown version is the printed menu. The AI doesn't need to inspect the decor. It needs the dishes and prices, without spending its limited context on navigation, scripts and repeated interface pieces.

Ben cohen

Which is where llms.txt comes in. Jeremy Howard proposed it. A markdown file at your site root, and llms-full.txt goes further with the flattened full content.

Ido

Think of llms.txt as a concise index and llms-full.txt as the whole menu with descriptions. But be careful. It's voluntary. It isn't robots.txt, it doesn't enforce anything, and nobody can promise it moves rankings. Support is real in documentation sites, like Anthropic, Perplexity and Zapier have published them, but broad adoption by AI search is unproven.

Ben cohen

So the honest answer to will this make me visible is no. It's one piece of readiness.

Ido

Same with schema. JSON-LD for organization, product, FAQ gives machines explicit context. But it must match what's visible on the page. Schema that says one price while the page says another is worse than none.

Ben cohen

And tools? LLMagnet is one example in this category. It tracks how you appear across ChatGPT, Claude and Gemini, runs prompt audits, and can manage llms.txt. Useful, but you can do the core of it by hand, and no tool fixes weak content.

Ido

Then there's the third audience. Humans, machines that interpret, and agents that act. An agent that understands your booking page but can't use the form leaves without buying. Clear labels, predictable flows, maybe APIs. That's likely the next layer after mobile and accessibility, though still early.

Chapter 2

What To Do This Week

Ben cohen

So what can someone do this week?

Ido

First, ask Claude, ChatGPT and Gemini your main buyer question. Check whether the description, pricing and specs match reality. Second, view the page source. If your core facts aren't in plain HTML, move them there.

Ben cohen

Then review crawler access in robots.txt, because blocking without understanding the consequences can make you invisible. Add accurate schema, consider llms.txt, and publish original material worth referencing. And don't stuff keywords for AI or mass produce thin pages.

Ido

And don't make the human experience worse for machines. Also, citations alone aren't success. Check accuracy and relevance too.

Ben cohen

My prediction, and it's only an opinion: within two or three years, serving a clean version of a page for machines will feel as normal as a mobile layout.

Ido

Mine is that agents completing tasks will become part of how we judge whether a site works. The goal isn't only visits. It's being discovered, understood, trusted, represented and eventually used correctly.

Ben cohen

Your small action: ask several AI systems the same customer question and compare. If you enjoyed this, follow Be Visible and share it with someone who owns a website.