A verified ISO automation build looks like a written list of acceptance criteria, checked one by one against the live system, with the pass or fail result recorded before anyone calls it done. On our reference build, that list ran to 19 items, and all 19 passed. Not "should work." Checked, one at a time, against the running system.
That distinction matters because "automation" is an easy word to say and a hard outcome to prove. Most vendors will tell you their system works. Fewer will show you the list they tested it against, and fewer still will tell you what else they checked to make sure nothing broke on the way.
What acceptance criteria mean for an ISO automation build
An acceptance criterion is a specific, checkable statement about what the system must do, not a vague goal like "handles revisions correctly," but something you can test and get a yes or no answer to. In ISO 9001 document control, that list has to cover the full shape of the workflow, not just the happy path.
For a document-control automation build, acceptance criteria typically span several categories:
- Lifecycle coverage — does the system correctly handle a new document, a revision, a cancellation, a controlled copy, and an uncontrolled copy, each one distinctly?
- Gate preservation — does every human decision gate in the existing SOP still function exactly as before, with the automation only acting after the gate closes?
- Field accuracy — when the system writes register data, is every field correct: revision number, effective date, amendment entry, distribution matrix, change-register row, request number, page count, retention date?
- Logging completeness — is every automated write captured individually in an activity log, so there's a record of exactly what the system did and when?
- Idempotency — if the same approval event fires twice, does the system produce one register entry or two?
- Non-interference — does anything else running on the same platform change behavior as a side effect of this build going live?
A build tested only on the first category — does it technically process a document — is not a verified build. It's a demo.
What we tested against, specifically
On our reference build, done for a Thai manufacturer certified to ISO 9001:2015 across two sites, the 19 acceptance criteria were derived from the manufacturer's own written SOPs, not a generic checklist. That's a meaningful difference. A generic checklist tests whether the automation behaves like automation in general. A criteria list built from your actual SOP tests whether it behaves like your process, specifically.
The system covers the full document lifecycle and preserves the four human decision gates already part of the manufacturer's process: a reviewer marking the paper change-request form, an approver checking content adequacy, a combined final gate completing master sign-off across internal, external, and support documents. Every one of those gates had to be verified intact before the build could be called complete — the automation had to act only after the gate closed, never before and never instead of it.
Once a gate closes, the system writes the eight fields a document controller previously retyped by hand: revision number, effective date, amendment-record entry, a master document list row spanning a ten-department distribution matrix, the change-register entry, the year-keyed request number, page count, and the calculated retention date. Each of those eight writes had its own line in the acceptance criteria list, checked individually — not "the register looks right," but "field seven, the retention date, calculates correctly against the retention policy."
Verifying no collateral damage
Passing your own criteria list tells you the new build works. It doesn't tell you whether building it broke something else. That's a separate check, and it's one a lot of automation projects skip.
On this build, the automation ran on a shared tenant with 51 pre-existing workflows unrelated to document control. After the new build went live, all 51 were verified unchanged — same behavior, same triggers, same outputs as before. That's a regression check on everything the new system was near, not a claim about the new system itself.
| What was verified | Result |
|---|---|
| Acceptance criteria (derived from the manufacturer's own SOPs) | 19 of 19 passed |
| Pre-existing unrelated workflows on the same tenant | 51 of 51 verified unchanged |
| Original source documents (file ID, name, format, modification time) | 5 of 5 verified unchanged |
Verifying the source stays clean
The third row matters as much as the first two. Five original source documents were part of this build's scope, and after go-live, all five were checked against their pre-build state and came back identical. An automation system that passes its own tests while quietly altering the documents it operates near isn't actually passing. It's failing at the one thing document control exists to protect: the integrity of the record.
Every automated write in this build is also logged individually to an activity log, and the workflow is idempotency-keyed — replaying the same approved event doesn't create duplicate register entries. Both of those are themselves acceptance criteria, not side benefits. Verification has to cover the boring, structural properties of the system, not just whether it produces the right-looking output on a good day.
Why this matters for your audit trail
An auditor who asks how a document-control automation was tested deserves a better answer than "we ran it a few times and it looked fine." A written list of acceptance criteria, checked individually with recorded results, is itself part of your evidence — it shows the system was verified against your actual process, not assumed to work because it worked somewhere else.
The number that matters isn't 19. It's that there was a list at all, that it came from your SOP rather than a template, and that every item was checked rather than assumed. Yours might need a different number. What shouldn't change is the discipline of writing the list down before you call anything finished.
FAQ
What does "19 of 19 acceptance criteria passed" actually mean?
A specific, written list of 19 testable requirements, derived from the manufacturer's own SOPs, was checked one by one against the live system, and every one passed. It's not a general impression that the system works.
Where do acceptance criteria come from?
From the client's existing SOPs and document-control process, not a generic template. That's what makes the list test whether the system behaves like your process specifically, rather than automation in general.
Does passing acceptance criteria guarantee nothing else was affected?
No. That requires a separate regression check. On this build, 51 pre-existing unrelated workflows on the same automation tenant were verified unchanged after go-live, in addition to the 19 acceptance criteria passing.
What happens if a criterion fails?
It gets fixed before the build is called complete. A criteria list only means something if failing items are treated as blockers, not noted and shipped anyway.
Does this replace user acceptance testing on our side?
No. Our criteria list is derived from your SOP and tested against the live system, but you should still walk through the workflow yourself before sign-off. The list makes that walkthrough fast and specific instead of open-ended.
If you want to see the actual 19-item list and what "unchanged" meant for the 51 workflows and 5 source documents, see the build record: the same build described here, with the verification results shown in full. 1% EVO builds and verifies this against your own SOPs before handing over a system you own outright.