Measuring 360° coverage instead of guessing it
"What did we test?" deserves a real answer. Here's how a coverage map turns a vague feeling into a number.

A common way to add "personas" to an AI testing agent is to put them in the prompt. "You are a 70-year-old shopper who isn't comfortable with technology." "You are a busy parent on a phone." Then you let the model act it out.
We tried thinking this through for our own design and kept hitting the same wall. Telling a model it's on a phone doesn't make the viewport 390 pixels wide. Telling it the network is slow doesn't make the images load late. The page the model sees is exactly the same page it would have seen without the costume. The persona is a story the model tells itself.
Worse, it's a story built on stereotypes. Ask a model to play an older user and it will often decide that means confused and slow, which tells you more about the model than about your customers.
So we split personas into two parts that behave very differently.
Everything about a persona that the website can actually detect goes into browser configuration, set by code before the session starts:
A rough sketch of what one looks like in our notes:
persona: returning-mobile-shopper
device: mid-range-android # 412 x 915, touch
network: slow-4g
locale: en-GB
timezone: Europe/London
account: returning-customer
behavior:
pace: hurried
scope: broad
typing: sloppy
None of the environment part involves the model. That matters for two reasons. It's real, so the bugs it finds are bugs real users on that setup would hit. And it's reproducible. When a finding comes in, the referee can replay it with exactly the same browser settings. You can't replay a mood, but you can replay a viewport.
The part of a persona that is about how someone uses the site does involve the agent. But instead of a character description, we give it positions on three axes.
Hurried vs. careful. A hurried user skims, clicks the first plausible button, and doesn't wait for the spinner. A careful one reads the shipping terms and checks the total before paying. Hurried users find race conditions. Careful users find inconsistencies.
Deep vs. broad. A deep session goes far into one flow: every option on the product configurator, every step of returns. A broad session touches many pages lightly. You want both, for different reasons.
Accurate vs. sloppy typing. This one is half code and half agent. Sloppy typing means a double space in the search box, a trailing space in the email field, a postcode in lowercase, a phone number with dashes. The agent decides where to type. Code decides how the typos are introduced, with a seed, so the referee can type the exact same mistakes again.
Three axes with two ends each gives eight combinations, which is a manageable number to rotate through, and every one of them maps to behavior you can describe precisely.
Prompt personas are more fun to read about. "Grandma tries to buy a gift card" makes a nicer demo than pace: hurried, typing: sloppy. We're fine with that trade.
It also fits the principle we wrote about last week: the model suggests, code decides. Here code decides what world the agent lives in, and the agent only decides what to do inside it.
We're still arguing about whether there should be a fourth axis. How someone reacts to an error, retrying stubbornly or giving up and leaving, is the leading candidate. If you've watched real users and think we're missing something obvious, tell us.
We're onboarding a small group of teams that ship with AI.
Get early access