Key takeaways
- In model-based testing (MBT), testers build an explicit model of expected behaviour, such as a state machine, workflow or grammar, and a tool generates tests from it according to a coverage criterion.
- The model is both a test generator and an oracle: it predicts what the system should do at every step. Changes to behaviour are made once, in the model, not in hundreds of scripts.
- It pays off for complex, long-lived, behaviour-rich systems, such as protocols, workflows and embedded control. It costs modelling skill and model maintenance, and only finds what the model can express.
Where this picks up
State transition testing already uses a model: you draw the states and derive tests by hand. MBT makes the model machine-readable and lets tools do the derivation, at a scale and rigour that hand derivation can't match.
How it works
- Model the expected behaviour at a chosen level of abstraction: states, actions, guards, expected outputs.
- Choose a coverage criterion on the model: all states, all transitions, all transition pairs, specific requirement tags or random walks with a stop condition.
- Generate abstract test cases: paths through the model, with expected results.
- Adapt: an adapter layer translates abstract actions ("login as admin") into concrete calls on the system (UI clicks, API requests).
- Execute and compare the system's actual responses with the model's predictions.
- Analyse failures: a bug in the system, the model or the adapter.
Offline vs online
- Offline MBT generates a test suite first, then runs it like ordinary automated tests. It's easy to review and repeat.
- Online (on-the-fly) MBT generates the next step while running, based on the system's actual responses. It's good for non-deterministic systems and long random walks.
What can be a model
| Model type | Good for | Example tools |
|---|---|---|
| State machines / graphs | Workflows, UIs, protocols, devices | GraphWalker, AltWalker |
| Executable specifications | Rich logic with data | Code-based models in the test language; stateful property testing |
| Process models (BPMN, activity diagrams) | Business processes | Various commercial MBT tools |
| Decision tables | Rule-heavy logic | Table-driven generators |
| Grammars | Parsers, file formats, protocols | Grammar-based fuzzers |
Stateful property-based testing is MBT in code: the model is a small program, and the generator explores random command sequences.
The abstraction decision
The most important design choice is what to leave out. A model of a banking app might include "balance positive / zero / negative" but not the exact amount; "logged in as customer / admin" but not every permission.
- Too detailed and the model becomes a second implementation: expensive, and buggy in the same ways.
- Too abstract and it can't predict outcomes precisely enough to catch real faults.
Good models capture what the business cares about and the risks you want to explore.
From 2002, as part of commitments to regulators, Microsoft had to document and verify the communication protocols used by Windows, so that other vendors could build interoperable software. The result was a very large body of technical documentation: more than 250 documents and around 30,000 pages, according to a 2011 Communications of the ACM article by Wolfgang Grieskamp, Nicolas Kicillof and colleagues.
The documents had to match the actual behaviour of Windows exactly. The team made model-based testing, using Microsoft's Spec Explorer tool, a cornerstone of its quality assurance process. Testers wrote models from the documentation, generated tests and ran them against the real implementations, so any disagreement exposed an error in either the documentation or the software.
Lesson: when the specification itself is the product, a model built from it, and tested against reality, is one of the few approaches that scales.
When MBT pays off
Good fit:
- Long-lived systems where behaviour changes often: update the model, regenerate the tests.
- Rich state and sequence behaviour, such as protocols, devices, workflows and stateful APIs.
- Standards and specifications that must be checked thoroughly.
- Teams with both domain knowledge and some programming skill.
Poor fit:
- Simple CRUD screens and one-off projects.
- Behaviour that's hard to abstract, like visual layout.
- Teams without time to learn modelling; a badly maintained model is worse than plain scripts.
Adopting it
- Pilot on one high-risk, stateful area, such as order lifecycle, a payment protocol or device mode switching.
- Model collaboratively with developers and product owners. Disagreements about the model are early bugs.
- Keep the adapter thin and stable; most churn should happen in the model.
- Version the model with the code, and review model changes like code.
- Measure the defects found, maintenance effort and coverage compared with your previous approach before scaling up.
Do
Treat the model as a shared specification: review it with the team, version it and keep it as small as the risks allow.
Don't
Model every detail of the implementation. You'll build a second system that has to be debugged and maintained alongside the first.
Check yourself
A generated test fails: the model predicts 'order status = Shipped' but the system says 'Packed'. What are the three possible explanations, and how do you decide?
Show answer
The system is wrong (a real bug), the model is wrong (the model misunderstood the requirement, or it changed), or the adapter is wrong (it mapped the action or the response incorrectly). Check the requirement or ask the product owner to decide between system and model, and inspect the adapter's logs to rule out a translation error.
