Measuring 360° coverage instead of guessing it

"Are we well tested?" is one of those questions everyone answers with a feeling. The feeling is usually wrong, and it's always unfalsifiable. Line coverage doesn't help much either: you can run every line of code and still never check whether the price the customer sees matches the price they're charged.

Testomatica replaces the feeling with a number. It does that with a coverage universe, an explicit map of everything worth testing, so "what did we cover?" becomes arithmetic instead of opinion. We wrote about the idea back in April. This week it stopped being an idea.

Four axes, one place for every test

Every test lands at a point on four axes. Together they describe not just that something was tested, but from which angle.

• Level: unit, component and integration, API contract, system (UI and HTTP), acceptance, and live monitoring.
• Surface: storefront, mobile web, APIs, admin and seller panels, background jobs, third-party integrations, the data layer.
• Quality dimension: function, data correctness, security, performance, accessibility, SEO, visual, device, usability.
• Lifecycle: requirement, design, PR, build, test, staging, production, hotfix.

Every item on the map gets an executor (code, an agent, or a person) and a result backed by evidence. An item left empty has a recorded reason.

Where we are today

On the marketplace we're piloting with, the map has 240 items. 154 of them, about 64%, are now guarded by code that runs on a schedule. Every remaining item is assigned to a planned human check, so nothing on the map is left without an owner. That second number matters more to us than the first.

Coverage that changes with the diff

Because files map to items on the map, a deploy's diff tells the planner exactly what just changed. Those items get tested first and explored hardest. The release report can then say something honest and specific: here's the change coverage, here's the weighted coverage, and here's the list of risks we did not get to this time.

Why "what we didn't test" is the valuable part

A good coverage model isn't there to produce a green 100%. It's there to make the gaps visible and name them out loud. Some things can't be observed from the outside; some can't be tested safely on staging. Those aren't failures. They're known blind spots, written down where a person can decide what to do about them.

That's the difference between a dashboard that flatters you and one you can actually plan with.