Answer Engine Optimization: What 13 Measured Websites Actually Do

Every agency in this market added AI search to its service list this year. Almost none of them measured anything.

So we did. On 1 August 2026 we ran a script against thirteen tour-operator homepages — one client of ours and twelve competitors in the same market — and recorded what each site actually hands a machine. On 12 August we re-ran the AI-specific checks to see whether anything had moved in eleven days. Nothing had.

This is what answer engine optimization looks like when you measure the field instead of describing it.

On what these numbers are. One request per site, from one location, on the dates given. It is a field survey, not a lab benchmark, and we will not present it as one. The competitors are anonymised deliberately: the measurements are of public pages and were taken fairly, but a named league table of somebody's rivals embarrasses third parties and starts an argument about method rather than a conversation about the work. The distribution carries the point on its own.

What we checked

Six things, all of them free to verify on any site in about a minute:

• Whether /llms.txt returns 200 — a plain-text summary of a site written for language models rather than crawlers
• Whether the homepage emits any JSON-LD structured data, and how many distinct types
• Whether robots.txt exists, and whether it names any AI crawler
• Whether robots.txt points to a sitemap
• Whether any analytics tag is detectable in the page source
• How long the homepage takes to arrive, paired with how much it weighs

The field, measured

The llms.txt and robots.txt rows were re-verified on 12 August 2026. The rest were measured on 1 August 2026.

Read the table whole and the story is not that these are bad businesses running bad websites. Nine of thirteen serve a fast homepage, which is more than you would expect. The story is that the machine-readable layer is close to empty — and that layer is the one an assistant reads.

Finding one: llms.txt is uncontested, and it is not a ranking factor

One site in thirteen publishes an llms.txt. Eleven days later it was still one.

We should be careful about what that means, because this is the exact point where the AI-search conversation usually stops being honest. No assistant has publicly committed to reading llms.txt. It is a proposed convention, not a standard, and it is not a ranking factor in any search engine. Anyone selling it to you as one is either guessing or hoping you will not check.

What it actually is: a cheap bet. It costs a morning to write and it puts a clean, unambiguous description of your business — services, locations, prices, the URLs that matter — in the one place a language model would look first if the convention holds. Ours runs to 11,933 bytes with a longer companion file at roughly 93 KB.

If the bet loses, you spent a morning. If it wins, you were first in your market by a year. That asymmetry is the whole argument, and it is enough. It does not need a fake statistic on top of it.

Finding two: naming GPTBot is not what grants it access

The most common piece of AEO advice in circulation is "add GPTBot to your robots.txt". On its own it is close to meaningless, and the measurement shows why.

A crawler reads robots.txt and obeys the most specific group that names it. If it is not named anywhere, it falls through to the User-agent: * group. So the question is never "did you name GPTBot" — it is "what does your catch-all say".

Eleven of the thirteen sites name no AI crawler at all. None of them are blocking anything. Their catch-all permits crawling, so GPTBot, ClaudeBot and PerplexityBot are already free to read every page. Two of the thirteen have no robots.txt at all, which is equally permissive. Eleven sites did nothing and got the outcome the twelfth paid an agency for.

Exactly one site in the field restricts an AI crawler by default. It is the one we built.

That site uses an allowlist: eighteen named agents — Google, Bing, DuckDuckGo, Apple, Yahoo, Yandex, OpenAI, Anthropic, Perplexity, Meta and the social preview bots — each given Allow: /, followed by a catch-all of User-agent: * and Disallow: /. Deliberate, internally consistent, and defensible as a way of turning away scrapers.

It also has a hole. Two tokens are missing from the list, so they land in the catch-all and get denied:

Be precise about what that costs, because overstating it would be the same sin as the advice we are criticising. Google-Extended governs whether your content can be used to ground Gemini and the Vertex AI generative APIs. Google states it is not a ranking signal and does not affect inclusion in Google Search, and AI Overviews follow Googlebot — which is named and allowed. So search visibility is untouched. The concrete loss is Gemini app grounding and Apple Intelligence, and today that is a small loss.

The pattern is the real defect. An allowlist denies every agent that has not been invented yet, and this field invents them monthly. A site whose entire positioning is being readable by machines should not have a default that says no.

We found this while writing this article, on our own work, eleven days after publishing a case study about that build. The two missing tokens went in the same day — they were an oversight, not a decision, since every AI agent the site's authors knew about was already allowed.

The larger question we have put to the client rather than answered ourselves: should the catch-all deny by default at all? Our own site does the opposite. And the case against an allowlist is that it buys less than it looks like it buys, because the scrapers anyone actually worries about ignore robots.txt entirely — the only crawlers an allowlist reliably turns away are the well-behaved ones. That is a policy call about how a business wants its content used, not a technical one, so it is theirs to make.

The lesson generalises further than either fix: decide your catch-all first. Everything else in that file is a detail.

Finding three: structured data is what actually earns the citation

Eight of thirteen homepages publish no JSON-LD at all. Search engines and assistants are inferring what those businesses are, where they are, and what they sell entirely from prose.

This is the finding that matters most and gets the least attention, because it is older and less exciting than llms.txt. When an assistant answers "who runs Muktinath pilgrimage tours from Kathmandu", it is assembling facts about entities: this organisation, that location, these services, that phone number. Prose is ambiguous and has to be interpreted. Structured data is not, and does not.

The honest caveat, since we are publishing the whole distribution: more is not better. The highest count in the field was 22 distinct types on a single homepage, and it is not ours — we would not copy it. Ours emits eight, and every one of them describes the business truthfully: TravelAgency, LocalBusiness, Organization, PostalAddress, GeoCoordinates, WebSite, SearchAction, EntryPoint. Schema you cannot defend in front of a human is a liability, not an advantage.

Finding four: eight of thirteen cannot tell you what worked

Five of thirteen run any analytics we could detect from the page source. The other eight are spending money on marketing with no way to know which spending produced anything.

It is not an AI-search finding, but it belongs here, because it is the reason most of this advice is unfalsifiable in practice. If you cannot see your baseline you cannot tell whether any of it worked, which means you will keep buying whatever was described most confidently. It is the single most expensive habit in this market, and it is why the first thing we ship on any engagement is measurement.

What we would actually do, in order

1. Fix the robots.txt catch-all. Free, takes ten minutes, and decides everything else. Add Google-Extended and Applebot-Extended explicitly while you are there.
2. Publish structured data that is true. Organization, LocalBusiness, the real postal address, real coordinates, real services. Fewer types described accurately beats twenty described loosely.
3. Answer the question on the page. Assistants quote text that answers a question directly. A page that states a price, a duration or a process in a plain sentence is quotable; a page of adjectives is not. This is most of the work, and it is the same work that has always earned rankings.
4. Then publish an llms.txt. After the three above, not instead of them.
5. Measure it. Baseline before you change anything, or you have no argument later.

Note that three of those five are ordinary technical SEO with a new audience. That is not a disappointment — it is the useful part. The overlap means the work pays off in Google whether or not the AI-search bet lands.

What we are not going to claim

Our own demand signal for this, from Search Console over the 30 days to 12 August 2026: the query ai seo expert nepal returned one impression, at position 80. By our own rule that is not a market. Nobody in Nepal is searching for this yet.

Assistant referrals are also badly attributed — GA4 sees a fraction of them, and a user who asks ChatGPT about you and then types your name into Google is recorded as direct or branded search. Anyone showing you a confident chart of AI-driven revenue is showing you an estimate.

So we are not selling AI search as a channel with proven returns. What the measurement supports is narrower and, we think, more useful: the entry price today is a morning's work, and twelve of thirteen sites in a competitive market have not paid it. That combination is rare, and when it stops being rare it will also stop being cheap.

We do this as part of SEO and AI-powered search work, and build the sites that make it possible under website and app development. The client in this study is a Kathmandu tour operator — one of the hospitality and travel operators we work with. If you want the same six checks run against your site and your three closest competitors, that is a fixed-scope audit and the output looks like the table above, with your names in it.

Book a 30-minute call and we will run it before we talk.