Key takeaways
- Verification: "Are we building the product right?" Does it match the specification?
- Validation: "Are we building the right product?" Does it meet the users' real needs?
- A product can match its spec perfectly and still be useless, if the spec was wrong. Testers help with both questions.
Two questions, two words
Software engineer Barry Boehm summed up the difference in two questions that testers still quote:
- Verification: Are we building the product right?
- Validation: Are we building the right product?
Verification compares the product with what was specified. Validation compares it with what people actually need.
Verification: matching the spec
Verification checks that a work product meets its specified requirements. It can be done on anything: requirements, designs, code or the finished system.
- Reviewing a design against the requirements.
- Code reviews and static analysis.
- Running test cases with expected results taken from the spec.
If the requirement says "passwords must have at least 12 characters", verification checks that an 11-character password is rejected.
Validation: meeting real needs
Validation checks that the product is fit for its intended use, by its real users, in its real environment.
- User acceptance testing with real users.
- Usability sessions: watching people try to complete real tasks.
- Beta releases and feedback from production.
- Prototypes shown to users before anything is built.
The 12-character rule might be verified perfectly. Validation asks whether users can still log in, or whether they now write passwords on sticky notes.
Side by side
| Verification | Validation | |
|---|---|---|
| Question | Built the product right? | Built the right product? |
| Compared with | Specification, requirements, standards | Users' real needs and use |
| When | Throughout development | Throughout, especially with working versions |
| Examples | Reviews, unit tests, system tests | Acceptance tests, usability tests, beta |
| Catches | Product doesn't match the spec | Spec doesn't match reality |
When verification says yes and reality says no
Hubble's main mirror was polished with extreme care, and measured again and again during manufacturing. Every measurement said it had the right shape.
After launch in 1990, Hubble's first images were blurry. The mirror was too flat near its edge, by about 1/50 of the width of a human hair.
NASA's investigation found the cause: the instrument used to check the mirror's shape had a lens 1.3 mm out of place. The mirror had been made to match a faulty reference. Other tests had shown signs of the error, but the results were set aside.
Astronauts installed corrective optics in 1993, and Hubble went on to do outstanding science.
Lesson: the mirror passed verification against its test instrument. Only real use, taking pictures in space, revealed it wasn't fit for purpose. Check your references, and take odd results from independent checks seriously.
What this means for testers
- Most test cases you write are verification: they check behaviour against requirements.
- Always keep validation in mind: "Would a real user be happy with this?" If the answer is no, raise it, even if the spec is met.
- Question the spec itself. A requirement can be clear, testable and still wrong.
Do
Report it when a feature matches the spec but would confuse or frustrate users. Explain the user impact and suggest a conversation with the product owner.
Don't
Stay silent because "it's what the requirement says". Requirements are written by people, and people make mistakes.
Check yourself
A tax calculation app produces exactly the results the specification requires. But the specification used last year's tax rates. Did verification pass? Did validation pass?
Show answer
Verification passed: the app matches its specification. Validation failed: the app isn't fit for its purpose, because users need this year's taxes calculated correctly.
