The Limits of TDD
My last post about TDD generated some great responses, some of which were skeptical. A few common complaints about TDD were brought up, and posed with civility, so I'd like to address them.
Complaint: You weren't stupid enough
When TDDing Fibonacci, we could get to a point where we have this function (and I did write exactly this code in my last post):
def fib(n):
if n == 0:
return 0
else:
return 1But why should we write that? Why not this instead?
def fib(n):
return [0, 1, 1][n]This comes down to how we define "simple". In TDD, we make tests pass by making the simplest possible change. So, which of the above two is simpler?
Defining that word is our job; TDD as a process says nothing about it. The definition is a huge variable and, in my experience, it's the primary axis along which our skill as TDDers grows once we reach minimal competence. Note that we still have to define "simple" even if we're not doing TDD, but we won't have the test-driving pressure forcing the definition to be refined.
Regardless of how "simple" is defined, we must eventually accept that an arbitrarily long list is not the simplest thing. At that point, we refactor. Depending on the definition of simple, it may take seven tests to get to the final refactor instead of five. So what? Two TDDers need not generate the same tests, and this isn't a problem at all.
Complaint: TDDed tests are prescriptive
This is a complaint that TDDed code does exactly what the tests say it should do, so there might be bugs. If I write the wrong test, the reasoning goes, then it will drive me to write the wrong code.
When would we write the wrong test? Only when we misunderstand the problem. If we misunderstand the problem, and we go straight to the code, then we're be encoding our incorrect understanding directly in the code. That's bad. By writing the tests first, we have some extra protection against misunderstanding: every assumption about what the system should do is encoded as a test, and each test has a good name.
Often, this will point out our confusion during the TDD process – we'll find that we want to write a test whose name contradicts another test's name. Even if we translate our misunderstanding into a bug, however, good test names make it easy to revisit our assumptions later. A subtle, five-character change to the code may have been driven by a sixty-character test name, which will be easier to understand.
Complaint: Choosing tests is hard
When TDDing Fibonacci, I tested fib(0) first. Why did I test fib(1) next instead of fib(37) or fib(51)?
Because it was obvious! The problem domain of a unit test is necessarily small, so it's usually clear what the next step is. If the next step isn't clear, it probably means that the unit under test is too large (making it hard to think about extending it for another case), or that we don't understand the problem well enough (making it hard to think about what the code should do at all). In either case, TDD has just helped us: it's either pointed out a bad design, which we should fix, or it's pointed a gap in our knowledge about the problem, in which case we should put the keyboard away and fill that gap.
Complaint: The code you TDDed was bad
The particular code I came up with in my last blog post was a slow, recursive Fibonacci solution. Two people mentioned this in the comments.
TDD doesn't solve problems like "my run time is superlinear" or "my database loads aren't eager enough." It's not supposed to solve those problems! TDD frees us to solve those hard problems well by (1) pushing us toward a good, decoupled design and (2) providing us with large, fast test suites.
Complaint: TDD requires too much typing
This one has the easiest answer of all: typing is not the bottleneck. Just think about it for a minute. Go back and look at how many lines of code you actually generated yesterday. How long would it take you to type it all in one long burst? A few minutes? Seriously, typing is not the bottleneck.
TDD is not magic
Let's recap:
- Complaint: You weren't stupid enough
- Response: There's more than one legitimate definition of "stupid".
- Complaint: TDDed tests are prescriptive
- Response: This is a feature. Stating our assumptions up front exposes misunderstandings.
- Complaint: Choosing tests is hard
- Response: This is also a feature. It tells us that our design is bad or that we don't understand the problem.
- Complaint: The code you TDDed was bad!
- Response: TDD does not free us from thinking. TDD is not magic.
- Complaint: It's too much typing.
- Response: Typing is not the bottleneck.
Many complaints about TDD are complaints that it doesn't solve some problem. These are not problems with TDD – it's not supposed to solve every problem!
Dynamic languages don't make coffee, continuous integration doesn't shine shoes, and TDD doesn't make code scale. It's simply the basis of a solid, disciplined process for building software – a beginning, not an end.