Software Testing

Expert path · lesson 7 of 21

ExpertTest design5 min readPublished November 12, 2026

Property-based testing: testing properties instead of examples

Instead of writing individual examples, you state rules that must hold for all inputs and let a tool generate hundreds of cases, then shrink any failure to its smallest form. Property-based testing finds the edge cases you didn't think of.

On this page

Key takeaways

  • A property is a statement that should be true for all valid inputs, such as "decoding an encoded value gives the original back". A tool generates many inputs and checks the property on each.
  • When a property fails, the tool shrinks the input to a minimal failing case, which often makes the bug obvious.
  • The hard part is finding good properties. Common patterns, such as round-trips, invariants, oracles, idempotence and model comparison, make it learnable. Stateful property testing extends it to whole systems.

Where this picks up

Partitions and boundaries help you choose examples, as covered in Equivalence partitioning and boundary values: beyond the basics. But you still choose them, so they reflect your assumptions. Property-based testing lets a machine explore the input space, guided by rules you state.

The approach was popularised by QuickCheck, introduced for Haskell by Koen Claessen and John Hughes in 2000. Today most languages have a mature library: Hypothesis (Python), fast-check (JavaScript/TypeScript), jqwik (Java), FsCheck (.NET), proptest (Rust) and more.

Example vs property

An example-based test:

def test_slug():
    assert slugify("Hello World") == "hello-world"

A property-based test with Hypothesis:

from hypothesis import given, strategies as st

@given(st.text())
def test_slug_properties(title):
    slug = slugify(title)
    assert slug == slug.lower()            # always lowercase
    assert " " not in slug                 # never contains spaces
    assert slugify(slug) == slug           # idempotent: slugifying twice changes nothing

The tool generates hundreds of strings: empty, very long, Unicode, emoji, control characters. If any fails, it shrinks the input. A failure on a 300-character string might shrink to "İ", a Turkish capital I whose lowercase form has two characters, revealing the real problem instantly.

Patterns for finding properties

PatternPropertyExample
Round-tripUndo after do gives the originaldecode(encode(x)) == x; parse(print(ast)) == ast
InvariantSomething is always preservedSorting keeps the same elements and length; a transfer keeps total money constant
IdempotenceDoing it twice equals doing it onceNormalising, deduplicating, applying the same config
Oracle / referenceMatches a simpler, trusted implementationOptimised search vs brute-force on small inputs
Hard to compute, easy to checkVerify the result instead of predicting itA sorted output is ordered; a computed route connects its endpoints
Commutativity / order independenceOrder of operations doesn't matter where it shouldn'tApplying discounts A then B equals B then A
MetamorphicRelated inputs give related outputsAdding an irrelevant item doesn't change a filter's result

Generators: the other half of the work

Properties are only as good as the inputs generated. Good generators:

  • Respect preconditions: generate valid orders, not random JSON that's rejected at the first check.
  • Bias towards edge cases: libraries already favour empty values, boundaries, zero, huge numbers and special characters.
  • Build structured data: users with orders with line items, consistent with each other.
  • Are composable, so complex domain objects are built from simple generators.

If most generated inputs are rejected by preconditions, the test explores very little. Watch the library's statistics.

Stateful property-based testing

The most powerful form tests sequences of operations against a model:

  1. Write a simple model of the system's state, such as a dictionary standing in for a key-value store.
  2. Define commands (put, get, delete) with preconditions and expected effects on the model.
  3. The tool generates random command sequences, runs them on both the real system and the model and compares results after every step.
  4. On failure, it shrinks the sequence to the shortest one that still fails.

This finds bugs that depend on specific orders of operations, which example-based tests almost never cover.

Real-world example · Quviq and Volvo Cars, AUTOSAR3,000 pages of specification as QuickCheck models

Car software built on the AUTOSAR standard must behave according to a very large specification. Working for Volvo Cars, John Hughes and colleagues at Quviq turned over 3,000 pages of AUTOSAR specification into QuickCheck state-machine models.

They then tested implementations of the standard from several software vendors with large volumes of generated test sequences. In their 2015 paper, they report raising over 200 issues with Volvo and the vendors. Hughes's talks on the project also describe many problems found in the standard itself, where the specification was ambiguous or inconsistent.

Lesson: a model plus generated sequences explored behaviour that no feasible number of hand-written tests could cover, and testing the model against reality also tested the specification.

Making it practical

  • Reproducibility: record the random seed for every failure, and save shrunk failures as regular example tests.
  • Budget: run a modest number of cases in CI (100–1,000 per property), and much longer campaigns nightly.
  • Readability: name properties as rules ("transfers preserve total balance"), so failures read like violated business rules.
  • Combine: keep a few clear example tests as documentation; let properties hunt for the edge cases.

Do

Write properties about what must always be true in the domain: "an invoice total always equals the sum of its lines plus tax, rounded once".

Don't

Re-implement the function inside the test to compute the expected value. You'll reproduce the same logic, and the same bugs.

Check yourself

Name three properties for a function `merge_sorted(a, b)` that merges two sorted lists into one sorted list.

Show answer

For example: the output is sorted; its length equals len(a) + len(b); it contains exactly the same elements as a + b, counted as a multiset. Also: merging with an empty list returns the other list unchanged, and merge_sorted(a, b) equals sorted(a + b) (a reference oracle).