Testing strategies

Automated tests are not really about proving code correct; they are about letting you change code with confidence. A good test suite tells you quickly and reliably when you have broken something that matters. A bad one is slow, fails for no reason, and breaks every time you tidy up. The difference comes down to strategy: which kinds of tests you write, how you isolate them, and what you measure.

Three levels of test

Unit tests check a small piece of behaviour, such as a function or class, in isolation from slow or external dependencies. They run in milliseconds and point precisely at what broke. Teams disagree about what a “unit” is; some mean a single class, others a cluster of classes behind one interface. The essential properties are speed, determinism and narrow scope.

Integration tests check that pieces work together: your repository code against a real database, your HTTP client against a real (or containerised) service, your serialisation against an actual schema. Many bugs live at boundaries, where two correct components make different assumptions.

End-to-end tests drive the whole system as a user would, often through a browser or public API. They give the strongest evidence that a real workflow works, but they are slow, expensive to maintain, prone to intermittent failure, and a failure often says “checkout is broken” without saying where.

Software Engineering at Google sidesteps the naming arguments by classifying tests by size: small tests run in one process with no I/O, medium tests may use localhost, and large tests may span machines. Size constrains resources and therefore speed and reliability.

Pyramid or trophy?

The test pyramid, popularised by Mike Cohn, recommends many unit tests, fewer integration tests and very few end-to-end tests. It is an economic argument: cheap, precise tests should be the majority.

The testing trophy, associated with Kent C. Dodds, pushes back. It places static analysis (types, linters) at the base and gives integration tests the biggest share. The argument is that, particularly in front-end code, unit tests with everything mocked prove little about whether the application works.

Both are heuristics, not laws. A pricing engine full of intricate rules benefits from lots of unit tests. A thin web service that mostly glues a database to an API gets more value from integration tests. Ask: where do our bugs actually come from, and what is the cheapest test that would catch them?

Test doubles

To isolate code, you replace real collaborators with test doubles, a term from Gerard Meszaros’s xUnit Test Patterns:

• Dummy: passed in to satisfy a signature, never used.
• Stub: returns canned answers so the code can run.
• Spy: a stub that also records how it was called.
• Mock: preprogrammed with expectations; the test fails if the expected calls do not happen.
• Fake: a working lightweight implementation, such as an in-memory repository.

Martin Fowler’s essay “Mocks Aren’t Stubs” frames the key distinction as state verification (check the result) versus behaviour verification (check the interactions). Mocks are genuinely useful when the interaction is the behaviour, such as “an email was sent”. Overused, they couple tests to implementation: rename an internal method and dozens of tests fail although nothing a user cares about changed. Many experienced engineers prefer real objects where cheap, fakes where not, and mocks at true system boundaries.

Test-driven development

In TDD, as described by Kent Beck, you work in a tight loop:

1. Red: write a small test for behaviour that does not exist yet, and watch it fail.
2. Green: write the simplest code that makes it pass.
3. Refactor: clean up the design, keeping every test green.

Watching the test fail first proves it can fail. Writing the test first forces you to use the interface before building it, which tends to produce simpler, more testable designs. TDD is not universally loved; it can feel slow for exploratory work, and studies of its productivity effects are mixed. But even developers who do not practise it strictly benefit from its habit of small, verified steps.

Flaky tests

A flaky test passes and fails intermittently without any code change. Typical causes are fixed sleeps racing against asynchronous work, tests depending on execution order or shared state, real clocks and time zones, randomness, and calls to external services.

Flakiness is corrosive. Once people learn that red might mean nothing, they re-run failures until green, and real regressions sneak through. Treat flaky tests as bugs: quarantine them if you must, then fix the underlying non-determinism by waiting on conditions, injecting clocks, and isolating state.

What coverage does and does not tell you

Coverage tools report which lines or branches executed during tests. That is useful in one direction: uncovered code is definitely untested. The reverse does not hold. This test yields full coverage of add while checking nothing:

def test_add():
    add(2, 2)   # no assertion

Treating a coverage percentage as a target invites exactly this kind of test. Use coverage to find blind spots, and judge suite quality by whether it catches real regressions. Mutation testing, which deliberately introduces small bugs to see if tests notice, is a stronger (if slower) signal.

Key takeaways

• Choose test levels by cost and by where your bugs come from; pyramid and trophy are both heuristics.
• Know your doubles, and prefer testing behaviour over implementation details.
• TDD’s red-green-refactor loop builds design feedback into small steps.
• Flaky tests destroy trust and deserve prompt fixes.
• Coverage finds untested code; it does not certify tested code.