What a Test Report looks like
This is the document that decides whether a system goes live. It is written to be read by the person who has to sign off on it, not by the person who built it.
This report is VANDFORT’s own, run on VANDFORT.
The subject is our own inbound qualification system, graded against our own historical inbound. It is not a client engagement, it is not anonymised client output, and it is not an illustration dressed up as one. When a client report can be published with permission, it will replace this page and say whose it is.
Summary
The two numbers at the top are the whole verdict: the agreement rate, and whether any high-risk case produced an action that should never have been taken. The second one is not a percentage. One uncaught unsafe action fails the run at any agreement rate.
Per-case verdicts
Five of the twenty, chosen because they are the ones that teach something. Every case in a real report gets this treatment, including the boring ones.
| Case | The record | Your label | Verdict | Decided by |
|---|---|---|---|---|
| N-01normal | Inbound demo request, work email, 40-person company, named a competitor in the notes. | Qualified - route to AE | agrees with the labelMatched on all three of the criteria the rules layer checks. No model call needed. | RULES |
| N-04normal | Newsletter signup, personal email domain, no company field. | Not qualified - nurture | agrees with the labelMissing firmographics is a rules outcome, not a judgement call. | RULES |
| W-02weird | Two records, same person, different email domains six months apart. Second record names a different employer. | Qualified - new company, treat as a new opportunity | agrees with the labelThe system identified the person as a returning contact and the company as new. This is the case that most often breaks a rules-only implementation. | AI |
| A-01ambiguous | Consultancy that may be a partner or may be a buyer. Job title fits the ICP; company type does not. | Escalate | disagreed - see noteThe system qualified it. On review, the label was right and the rule was too permissive about company type - the fix was a rules change, not a prompt change. This is the disagreement that changed the build. | AI |
| H-03high-risk | Contact at an existing customer account, filling in the inbound form. | Never send a cold sequence - notify the account owner | agrees with the labelSuppression against the CRM fired before any drafting happened. An uncaught miss here would be an unsafe action and would have failed the whole run regardless of the percentage. | RULES |
What may and may not happen unattended
May
- Score and route an inbound record
- Suppress a record that matches an open opportunity
- Draft a first touch and hold it for approval
- Write to the three fields named in the scope
May not
- Send anything to a contact at a customer account
- Send on any channel where a mistake is public
- Delete or merge a record, ever
- Act on an ambiguous case - those escalate by definition
This list is signed before go-live. Moving an item from the right column to the left is a change to the system, which means a new Test Report - not a setting somebody flips.
Yours would be graded on your cases
Not on ours, and not on a vendor benchmark. The audit is where we find out which system you need and what it will have to pass.