Agent readiness

We scanned 652 small-business websites. 31% are invisible to AI agents.

Original data from a neutral, reproducible sample of real SMB websites across restaurants, health, home trades, personal care, and independent retail. Nearly a third offer AI agents nothing machine-readable at all, and not a single site in the sample exposes a surface an agent could act on. Here is the data, industry by industry, with the full methodology so you can check our work.

Updated August 10, 2026 · 11 min read

AI assistants now recommend, book, and buy on behalf of their users, and the sales they influence are measured in the hundreds of billions of dollars. So we asked a question with a measurable answer: if an AI agent visits an ordinary small-business website today, what can it actually read? We scanned 652 real SMB websites across five industries with the same deterministic scanner that powers our free /scan tool. The headline: 30.7% are invisible to agents, meaning they offer no machine-readable facts and no agent artifact of any kind. And on the question of whether an agent could act rather than just read, the number is not low, it is zero: not one site in the sample exposes a callable agent surface.

This article reports what we found, industry by industry, and then shows the full methodology: where the sample came from, how sites were selected, what the scanner checks, and what this study cannot tell you. Every number below is reproducible from the stored per-site results.

The headline numbers

• 30.7% of sites are invisible to agents: unreachable at scan time, or no valid structured data and no agent artifact of any kind (defined precisely below).
• 38.8% have no valid JSON-LD structured data, the single highest-leverage artifact an agent consumes.
• 100% expose zero callable agent artifacts. Across all 652 sites we found no agent.json, no MCP server card, and no OpenAPI spec. Not one.
• 4.1% publish machine-readable pricing, even though 22.7% show a price somewhere a human can see it.
• 8.9% returned no successful response to our honestly identified scanner (hard 403/503 blocks or outages), and a site that refuses an identified bot refuses real agent traffic too.
• 5.1% block at least one major AI crawler in robots.txt, and 33.0% serve an llms.txt file, a number with an interesting explanation covered below.

The mean readiness score across the sample was 49.1 out of 100 (median 51). For calibration: a site scores in that range when it is reachable over HTTPS with normal semantic HTML and perhaps builder-generated structured data, but nothing typed about its offers and nothing callable. Human-readable, machine-opaque.

What "invisible to agents" means, precisely

Composites are where studies cheat, so here is ours in full. A site counts as invisible to agents when either (a) its homepage was not reachable with a successful response at scan time, or (b) it has no valid, parseable JSON-LD structured data AND none of the six agent artifacts the scanner probes for: agent.json, a well-known agent.json, an A2A agent card, an MCP server card, an OpenAPI spec, or an llms.txt file. In plain terms: the site gives an agent no typed facts and no machine-readable surface at all. An agent visiting it can only scrape prose, which is exactly the failure mode that makes assistants hallucinate prices and hours or skip a business entirely.

The definition is deliberately generous, and the generosity is measurable. We count llms.txt as an artifact in a site’s favor even though it has no documented consumers, and 8.6% of sites qualify as visible only because of it. Remove llms.txt from the definition and the invisible share rises from 30.7% to 38.8%. We report the generous number as the headline and the strict one here, so nobody has to take our framing on faith.

The per-industry breakdown

Three patterns stand out. Home trades lead on transactability signals by a wide margin (18% publish some structured offer schema and 13% structured pricing, against low single digits everywhere else), consistent with trade-focused site builders baking LocalBusiness and Service markup into their templates. Independent retail is the structured-data laggard, with 47% lacking any valid JSON-LD despite being the vertical closest to e-commerce. And health is the least invisible vertical (28%) while publishing machine-readable pricing almost never (5%), a familiar opacity that will serve those practices poorly when patients start asking assistants what an appointment costs. Restaurants and personal care tie for the most invisible at 33%.

The transactability gap

Discovery is only half of agentic commerce; the other half is whether an agent can act. Here the numbers fall off a cliff. 58.4% of sites show a human-clickable action (book, order, request a quote), and 31.9% expose that action somewhere in structured data, but the artifacts an agent could actually call are simply absent: 0% serve an agent.json, 0% an MCP server card, 0% an OpenAPI spec, across the entire sample. The pattern is unmistakable: Main Street has built for human eyes and left machines to guess. In practice, when a customer asks an assistant to "book me a plumber Tuesday afternoon," the assistant cannot complete that request against any of these 652 sites; at best it summarizes a phone number. The businesses that close this gap first inherit that intent, and the mechanics are covered in how AI agents book service businesses.

Robots.txt and the llms.txt surprise

Crawler blocking turned out to be modest and oddly targeted. 5.1% of sites block at least one major AI token in robots.txt, and the blocking skews heavily toward training crawlers: GPTBot (4.9%), ClaudeBot (4.6%), and Google-Extended (4.1%) are the usual targets, while the agents that fetch pages on behalf of a live user request (ChatGPT-User, Claude-User, Perplexity, the search bots) are blocked by fewer than 1% of sites. That pattern looks like anti-training defaults spreading through CMS plugins rather than deliberate anti-agent policy, and it means the blocking is mostly symbolic today: a site that blocks GPTBot still serves an agent acting for a real customer. The self-inflicted damage sits elsewhere, in the 8.9% of sites whose firewalls returned hard 403 or 503 responses to an honestly identified scanner.

The genuine surprise was llms.txt at 33.0%, an order of magnitude above what organic adoption would predict. The likely explanation is not a grassroots movement: major website builders began auto-generating llms.txt for every site they host in 2025, and 24.4% of our sample serves llms.txt alongside valid builder-style JSON-LD, the signature of platform defaults rather than deliberate choice. Read against our earlier llms.txt analysis, the takeaway is unchanged: the file is cheap infrastructure that platforms now ship automatically, while the artifacts with documented consumers still depend on the business doing real work.

Methodology

The sample frame is OpenStreetMap, queried through the public Overpass API on August 10-11, 2026. OSM is a public commons directory rather than a marketing-selected or self-submitted list, and the exact queries are code in our repository, so the frame is reproducible. We drew from 12 fixed, region-diverse mid-size US metros chosen in advance (Columbus OH, Raleigh NC, Tucson AZ, Spokane WA, Grand Rapids MI, Chattanooga TN, Boise ID, Worcester MA, Baton Rouge LA, Reno NV, Des Moines IA, Richmond VA) across the five verticals in the table above.

• Eligibility: the OSM listing must link the business’s own website. Platform pages (Facebook, Instagram, link hubs, ordering platforms) were excluded, and chains were excluded via OSM brand tags, because the study measures independent businesses’ own sites.
• Selection: within each metro-vertical cell, eligible domains were ordered by a cryptographic hash seeded on the cohort label and capped at 14 per cell. No manual picking anywhere in the pipeline; domains were deduplicated across the sample.
• Scanning: each site’s homepage was fetched once with the same deterministic scanner (version 2) that powers the public /scan, which also probes robots.txt, llms.txt, agent.json, well-known agent endpoints, an MCP server card, and an OpenAPI spec. No LLM is involved in scoring; every check is a reproducible boolean.
• Politeness: before fetching any page, the harness checked each site’s robots.txt for its own honestly named user agent and skipped sites that disallowed it (5 sites were excluded this way and are not in any denominator). Scans ran in small batches spread over hours.
• Accounting: 722 sites were sampled, 655 completed scans, and 62 failed after three attempts (unresolvable hostnames, timeouts); failures are excluded from all percentages. After deduplicating redirect targets by hashed domain, 652 unique sites form the reported dataset.

Why this gap exists, and why it will not last

None of these numbers reflect laziness. Ten years of SEO advice taught businesses to write for Google’s crawler, and Google could always compensate for messy markup with scale. Agents are different consumers: they need typed facts to trust (a price field, not a price mentioned in a paragraph) and callable surfaces to act (a bookable endpoint, not a phone number in a footer). The tooling to publish those artifacts has existed for years (JSON-LD is a copy-paste job for most sites), but nothing forced adoption. Agent-mediated purchasing is that forcing function, and the current state of agentic commerce rewards early movers precisely because the baseline is this low.

The competitive read is simple. In a sample where roughly a third of local competitors are invisible and none are transactable, being merely legible puts a business in the upper tier of its market, and being transactable, where an agent can check availability and book or complete a checkout, is literally uncontested: zero sites in 652 offer it. That is a strange, temporary arbitrage, and it is measurable: we will rerun this exact methodology on a fresh cohort in six months and publish the trend.