The only kind of test I write

There is a near constant background-level conversation in the engineering world about which kinds of tests are better; unit tests, integration tests, end-to-end tests, and so on.

A while back, I decided I was only going to write one kind of test.

I’m not really sure it neatly fits into any of the categories above. Perhaps ‘integration test’ is most correct, but that doesn’t quite cover it — not least because nobody can really agree on how to actually define what these different labels mean.

Here are the principles I follow:

1. I don’t write a single test until I have a clear public interface defined.
2. My tests must emulate as closely as possible the behavior of an actual user. That means calling public apis only, or if it’s a front-end interface designed for an end-user, doing only what an end-user would.
3. If a function or component is not part of the public interface, I don’t directly test it. If it’s not possible to invoke it indirectly via some public path, what exactly is its purpose anyway?
4. No remote network calls. If it’s not a call to localhost (for instance, a local test database) then it needs to be mocked. My tests should pass consistently with no network or internet connection.
5. No mocking of internal code paths. Mocking at the network boundary is great, but mocking functions or apis that exist within the codebase I’m testing is a no-go.
6. No state carried across different tests. If a test is relying on the state from a previous test, or could be affected by that state in any way, something has gone wrong. This doesn’t have to mean ‘clear all state after each test’.
7. No testing of internal state. If I have to initialize or tweak some internal state by hand in my test (without using another existing public interface), my test is probably not a good one.
8. No testing of internal implementation details. If refactoring my code will make the test fail, it’s probably not a good test.
9. No ‘unit tests’ that invoke internal functions. If I feel the strong desire to test functions like this, they should probably be broken out into their own standalone module or package beforehand, with a clear interface that is completely decoupled from any of my business logic.
10. No ‘end-to-end’ style tests that require spinning up a full server and database and multiple services and running them all together, unless it’s extremely expedient to do this on my localhost.
11. I focus on ‘coverage of use-cases’ rather than ‘coverage of lines of code’.
12. Write a test first if it’s very clear what the public interface will look like in advance. Otherwise write the code first, then test it.
13. I break any of the above rules when it’s better to do so than to not write a test or to leave a use-case untested.