How to verify an AI agent is who it says it is

Every allowlist published in the last two years rests on two things a stranger can forge or a vendor can change without telling you: a user agent string and an IP range. Both have now run out of road for the traffic that matters most. The replacement is a signature you can check, and it is already arriving at your origin.

Reviewed September 15, 2026 · 10 min read

Every piece of advice about AI bots, including ours, has told you to write rules against the user agent string and then verify against a published IP range. That advice was correct in the sense that it was the best available. It was never good. The string is a field an attacker types, and the IP list is a file a vendor maintains at its own convenience.

That approach has now failed outright for the single most valuable class of traffic. OpenAI’s own allowlisting guidance for its agent does not publish IP ranges at all, and does not name a user agent string to match on. If your access policy is built on those two inputs, you have no supported way to recognise an agent that is standing in your checkout on behalf of a real customer.

What replaced it is a cryptographic signature on the request itself, with a public key directory you can open in a browser right now. This guide covers what the signature carries, what you can check yourself, the part almost nobody has noticed yet, and what is honestly worth doing this week.

Why the allowlist you already published is a guess#

Two mechanisms carried the whole industry until recently, and it is worth being precise about how each one breaks, because the failures are not symmetrical.

The last row is the one that changed the calculus. Crawlers mostly still publish ranges. Agents, which run inside a cloud browser on behalf of a person who is mid-task, largely do not, because the whole point of that infrastructure is that it is elastic. An agent doing your customer’s shopping is exactly the request you least want to refuse, and exactly the one your existing rules cannot place.

What a signed request actually carries#

Web Bot Auth applies HTTP Message Signatures, RFC 9421, to bot and agent traffic. Three headers arrive together:

• Signature, the signature bytes themselves.
• Signature-Input, the metadata describing what was signed and under which parameters.
• Signature-Agent, a URL identifying who is claiming to send this. OpenAI sets it to https://chatgpt.com.

The architecture draft is specific about the parameters, and the specifics are what make this worth more than the string it replaces. The keyid must be a base64url JWK SHA-256 thumbprint, so the key identifies itself rather than being a label someone chose. The tag must be the literal value web-bot-auth, which is how a verifier knows the signature is asserting bot identity rather than something else. The request authority must be among the signed components, so a signature captured against one host cannot be replayed against yours. And created and expires are carried in Signature-Input, so a signature lifted off the wire stops working shortly afterwards.

The directory is the part you can check yourself#

A signing operator publishes its public keys as a JSON Web Key Set at a well-known path on its own domain: /.well-known/http-message-signatures-directory. A verifier reads the Signature-Agent URL, fetches that path on it, finds the key whose thumbprint matches keyid, and checks the signature. Cloudflare supports the Ed25519 algorithm for this, and accepts every valid Ed25519 key it finds in an operator’s directory.

This is unusually easy to satisfy yourself about, which is rare in this field. OpenAI documents its directory location openly, and you can fetch it the same way a verifier would:

curl -s https://chatgpt.com/.well-known/http-message-signatures-directory

# Returns a JSON Web Key Set. Each key in it can sign requests
# that arrive at your origin carrying Signature-Agent: https://chatgpt.com

There is no registration, no partnership and no account required to look. The trust model is deliberately the same one the web already uses for certificates: the claim is anchored to a domain, and the domain publishes the material that proves it.

The agent card nobody has read yet#

The second draft in this family is the one with the most operational value and the least attention. Alongside the key directory, an operator can publish a signature agent card: a JSON document describing what the agent is and how it intends to behave. It borrows OAuth client metadata for the identity half, client_id, client_name, client_uri, logo_uri, contacts, jwks_uri, and adds a web_bot_auth object for the behaviour half.

Read that table again with the robots.txt file you currently publish in mind. Every line in that file is a product token you found in a blog post, copied, and hoped was current. The card is the operator stating it directly, next to the key that proves the operator is who it says. The guessing game that has defined crawler management since 1994 has a lookup now.

Fetcher or crawler is the distinction that pays#

If you take one operational idea from this, take the trigger field. A crawler is collecting at its own initiative, for training or for an index. A fetcher is retrieving this page, right now, because a person asked a question and is waiting. Those two deserve opposite default treatment, and until now you had no reliable way to tell them apart, which is why so many sites ended up blocking both or neither.

Blocking a crawler costs you presence in an answer you might have been cited in. Blocking a fetcher costs you a customer who is standing at your door with their wallet out, and it costs you silently, because nobody files a support ticket about a shop they could not get into. Which crawlers to allow works through the roster and the policy; the card is what finally makes that policy enforceable against something other than a string.

What to actually do this week#

The honest answer for most businesses is that you are not going to implement RFC 9421 verification, and you should not. Your edge provider either does this or does not, and that is the decision in front of you.

1. Find out whether your CDN verifies signatures today. If you are on Cloudflare, validation happens at the edge automatically and verified traffic is marked as such, with cf.bot_management.verified_bot populated. If you are elsewhere, ask the question directly rather than assuming.
2. Write your rules against your provider’s bot name, not a user agent. These differ per provider for the same agent: Akamai calls it ChatGPT Agent, Cloudflare calls it chatgpt-agent, Vercel calls it chatgpt-operator. A rule copied from a Cloudflare tutorial into an Akamai config matches nothing and fails open or closed without telling you.
3. Audit what your edge already blocks before touching robots.txt. Bot fight modes and AI scraping toggles operate above robots.txt, and a welcoming file cannot un-refuse a request the edge already dropped.
4. If you do terminate at your own origin, verification is available rather than theoretical. Cloudflare’s open-source repository ships working verifiers for Workers, a Caddy plugin and Rust, along with the thumbprint and directory logic. It also states plainly that it has not been audited.
5. Leave the IP allowlists you have in place. Nothing here breaks them, and crawlers that publish ranges still publish them. Just stop treating them as the plan for agent traffic, because for the ChatGPT agent there is no list to treat that way.

What is genuinely not settled#

It would be dishonest to present this as finished. The registry and agent card specification is at revision 03, dated June 2026, and it is an individual Internet-Draft. It carries the standard language: not endorsed by the IETF, no formal standing in the standards process. It has not been adopted by a working group. The authors are from Cloudflare and Amazon, which tells you both that it has real implementation weight behind it and that it is not yet a neutral community product.

So field names can change, and a card you parse today may need revisiting. That is an argument for reading cards rather than building products on them, which is what almost everyone reading this should be doing anyway.

The deeper limit is one no protocol solves: a signature proves identity, not virtue. It tells you the request is genuinely from the operator it claims, and nothing whatsoever about whether that operator will respect your rate limits, honour your robots.txt, or use what it collects the way its card says. Identity is the precondition for accountability, not a substitute for it. What it buys you is that when an agent misbehaves, you now know precisely whose behaviour it was, which is more than the last two years of allowlists ever gave you.

The practical posture, then, is unglamorous: stop investing in string matching, find out what your edge provider verifies, and get comfortable with the idea that agent traffic will soon carry proof of origin the way HTTPS carries proof of server. If your catalog and prices are machine-readable and your identity resolves cleanly, the agents that prove who they are will be able to act on all of it.