!title The Three Laws of TDD. Over the years I have come to describe Test Driven Development in terms of three simple rules. They are: 1 You are not allowed to write any production code unless it is to make a failing unit test pass. 2 You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures. 3 You are not allowed to write any more production code than is sufficient to pass the one failing unit test. You must begin by writing a unit test for the functionality that you intend to write. But by rule 2, you can't write very much of that unit test. As soon as the unit test code fails to compile, or fails an assertion, you must stop and write production code. But by rule 3 you can only write the production code that makes the test compile or pass, and no more. If you think about this you will realize that you simply cannot write very much code at all without compiling and executing something. Indeed, this is really the point. In everything we do, whether writing tests, writing production code, or refactoring, we keep the system executing at all times. The time between running tests is on the order of seconds, or minutes. Even 10 minutes is too long. Too see this in operation, take a look at [[The Bowling Game Kata][TheBowlingGameKata]]. Now most programmers, when they first hear about this technique, think: "''This is stupid!''" "''It's going to slow me down, it's a waste of time and effort, It will keep me from thinking, it will keep me from designing, it will just break my flow.''" However, think about what would happen if you walked in a room full of people working this way. Pick any random person at any random time. A minute ago, all their code worked. Let me repeat that: ''A minute ago all their code worked!'' And it doesn't matter who you pick, and it doesn't matter when you pick. '''A minute ago all their code worked!''' If all your code works every minute, how often will you use a debugger? Answer, not very often. It's easier to simply hit ^Z a bunch of times to get the code back to a working state, and then try to write the last minutes worth again. And if you aren't debugging very much, how much time will you be saving? How much time do you spend debugging now? How much time do you spend fixing bugs once you've debugged them? What if you could decrease that time by a significant fraction? But the benefit goes far beyond that. If you work this way, then every hour you are producing several tests. Every day dozens of tests. Every month hundreds of tests. Over the course of a year you will write thousands of tests. You can keep all these tests and run them any time you like! When would you run them? All the time! Any time you made any kind of change at all! Why don't we clean up code that we know is messy? We're afraid we'll break it. But if we have the tests, we can be reasonably sure that the code is not broken, or that we'll detect the breakage immediately. If we have the tests we become fearless about making changes. If we see messy code, or an unclean structure, we can clean it without fear. Because of the tests, the code becomes malleable again. Because of the tests, software becomes soft again. But the benefits go beyond that. If you want to know how to call a certain API, there is a test that does it. If you want to know how to create a certain object, there is a test that does it. Anything you want to know about the existing system, there is a test that demonstrates it. The tests are like little design documents, little coding examples, that describe how the system works and how to use it. Have you ever integrated a third party library into your project? You got a big manual full of nice documentation. At the end there was a thin appendix of examples. Which of the two did you read? The examples of course! That's what the unit tests are! They are the most useful part of the documentation. They are the living examples of how to use the code. They are design documents that are hideously detailed, utterly unambiguous, so formal that they execute, and they cannot get out of sync with the production code. But the benefits go beyond that. If you have ever tried to add unit tests to a system that was already working, you probably found that it wasn't much fun. You likely found that you either had to change portions of the design of the system, or cheat on the tests; because the system you were trying to write tests for was not designed to be testable. For example, you'd like to test some function 'f'. However, 'f' calls another function that deletes a record from the database. In your test, you don't want the record deleted, but you don't have any way to stop it. The system wasn't designed to be tested. When you follow the three rules of TDD, ''all your code will be testable by definition!'' And another word for "testable" is "decoupled". In order to test a module in isolation, you must decouple it. So TDD forces you to decouple modules. Indeed, if you follow the three rules, you will find yourself doing much more decoupling than you may be used to. This forces you to create better, less coupled, designs. Given all these benfits, these stupid little rules of TDD might not actually be so stupid. They might actually be something fundemental, something profound. Indeed, I had been a programmer for nearly thirty years before I was introduced to TDD. I did not think anyone could teach me a low level programming practice that would make a difference. Thirty years is a ''lot'' of experience after all. But when I started to use TDD, I was dumbfounded at the effectiveness of the technique. I was also hooked. I can no longer concieve of typing in a big long batch of code hoping it works. I can no longer tolerate ripping a set of modules apart, hoping to reassemble them and get them all working by next Friday. Every decision I make while programming is driven by the basic need to be executing again a minute from now. ---- !* Fri, 7 Oct 2005 03:37:42, Johan Nilsson, Bug fixing This might be a bit off topic, but how about adding 'You are not allowed to fix any code without having a reproducible (unit) test case'? OK, does not apply to the initial development, but bugs do pop up even in systems developed using TDD (even though the are rare when applied correctly). Or is this TDBF - 'Test Driven Bug Fixing' ;-) *! !* Fri, 7 Oct 2005 08:16:14, Craig Demyanovich, Bug Fixing For my current project, still in development, we do create failing tests to correct mistakes. Using the ideas Uncle Bob presents allows us to release often. Invariably, there are some minor things that we didn't do well, or there are some cases of some story that neither the customer nor the developers considered. We take that feedback, create a failing test for each item and make it pass. We immediately demonstrate the correction to the customer, who knows that the problem will not be present in the next release. So, Johan, I see no reason why this technique can't apply throughout the entire lifetime of the application. *! !* Fri, 7 Oct 2005 19:47:51, Justice, What about UI? So, what do you do for user interface code (e.g., on a web application)? I know that quite a bit of it can be decoupled, but not all of it... *! !* Sat, 8 Oct 2005 04:51:40, , *! !* Sat, 8 Oct 2005 05:05:30, Sagy Rozman, TDD for UI code There are lots and lots of good examples of web apps written using TDD. You can check the different yahoo discussion groups related to this topic. Even better - download the Fitnesse source, it's a web application. I haven't checked it, but I bet you can find some good TDD examples for the UI modules since it was (mostly?) written by Robert Martin himself. * ''It was mostly written by Micah Martin, but I helped a bit. == UB.'' *! !* Sat, 8 Oct 2005 05:10:08, Sebastian Kübeck, There are more things that cannot be unit tested... http://www.artima.com/weblogs/viewpost.jsp?thread=126923 I think the rules have to be reduced mostly to the domain logic to be feasable. ''All those things can (and should be) tested, the only issue is whether you count those tests as unit tests or not. -- Michael Feathers'' *! !* Sat, 8 Oct 2005 11:46:17, Sebastian Kübeck, > All those things can (and should be) tested, > the only issue is whether you count those tests as unit tests or not. -- Michael Feathers The rules are explicitly about unit testing. I think the trick is to keep code that's not unit testable as small as possible and to introduce automated and reproducable tests for code that e.g. talks directly to the database. A replacement (e.g. in memory data store) for the code that is not unit testable is necessary to keep the unit tests quick an independent of the outside world. *! !* Mon, 10 Oct 2005 14:33:05, Uncle Bob., Code that's not unit testable. I have another rule. It's not part of the three, but it is part of my personal repertoire. ''Rule 4: There is no such thing as untestable code.''. Is this rule true? I don't know for sure. However, I treat it as an attitude. I refuse to believe that, given appropriate creativity and imagination, there is some code that cannot be tested. Indeed, I refuse to believe that, given appropriate creativity and imagination, there is code that cannot be ''profitably'' tested. Accepting that there are certain kinds of code that are "impossible" to test is a slippery slope. It's an excuse to abandon creativity, and submit to the siren song of the schedule that whispers that you don't have time, you don't have time, you don't have time. If you have code that you believe cannot be tested, then I suggest that you: 1 Stop, and find a way to test it. 2 Find a way to change it so that it's testable. *! !* Tue, 11 Oct 2005 09:44:13, Sebastian Kuebeck, Great point! I'd make Rule 4 actually Rule 1 since it's the most important! I absolutely agree with your opinion that everything should be tested automatically. The point is the time a test takes to run. I keep slow tests (e.g. calls to stored procedures that need to shouvle around some data between a number of tables) in a different directory than the quick ones that drive my development so I'm not blocked by slow infrastructure during development (TDD is a different word for green arrow obsession ;-) ). I call the slow ones also frequently, especially before or during I take a break. Maybe I'm wrong but I thought only the quick ones where called unit tests and the slow ones integration or whatever tests, that's all. *! !* Tue, 11 Oct 2005 19:43:05, Keith Gregory, What if the next logical test requires more than 10 minutes of coding? I like the idea of TDD, and firmly believe that "if it's difficult to test, it will be difficult to use." However, I feel that I must be missing a deeper insight, because there seem to be many cases where the implementation of an object does not grow linearly with tests. For example: this weekend I tried to implement a Skip List using the TDD rules from this article. Skip List is a replacement for binary trees, which uses a random number generator to distribute entries (see http://en.wikipedia.org/wiki/Skip_list). The external API is simple, generally following other Java collections. So, a reasonable series of tests for this API seem to be: 1) Construct, 2) Add object, 3) Check Size, 4) Retrieve object. Following this sequence, however, there's a huge implementation gap between tests 1 and 2: the whole internal management of entries. I can of course defer that implementation, perhaps using a single-level linked list with insertion to get all the way to test 4, but that doesn't really take me in the direction of a skip list. Most of the "interesting" implementation happens in an inner class and a couple of private methods in the outer class, and I don't want to expose those just for the purpose of testing. A slightly different point is verification that my implementation actually builds a series of linked lists that have the correct distribution. Testing this fact is critical to knowing that I have an object that will store/retrieve in O(log2N) time versus O(N) time. My only answer within the bounds of "test the interface" is to add a distribution method to the interface. Perhaps useful for other purposes, but it exists for testing. *! !* Thu, 13 Oct 2005 13:42:10, George Dinwiddie, untestable code ''"I have another rule. It's not part of the three, but it is part of my personal repertoire. Rule 4: There is no such thing as untestable code."'' Hmmm... I think I've seen untestable code. I would change Rule 4 to Commandment 1: ''Thou shalt not write untestable code.'' It's not that code that does certain things is untestable, but that people write it in a way that's untestable. Back in the early 1980s, when I was working in hardware development, there was a mantra in the trade magazines, "Design for testability." This included things such as having or being able to set a known starting state for a circuit, and having access to sufficient internal nodes to determine proper operation. As hardware development moved from discrete circuits to custom integrated circuits, these issues came to the fore because a technician was more limited in the transformations he could make to allow testing. The same issues face the software development community. *! !* Thu, 13 Oct 2005 19:20:53, Moz, So how do I test UI code? > If you have code that you believe cannot be tested, then I suggest that you: > 1. Stop, and find a way to test it. > 2. Find a way to change it so that it's testable. A huge amount of what I do is UI or interface code. Making sure that (eg) drag and drop works properly is interesting and tricky to test, but it's nothing compared to some of the more intense UI code. Call this method, watch it alter a spreadsheet... some kind of test to make sure that all the values are correct is necessary, but a bit outside the scope of most testing systems. Currently we're using TestComplete and screenshots, but to say that that sucks is an understatement. The user enabling font aliasing is a slightly different error to all the values in a spreadsheet being wrong, but the test result is the same... So I wonder how people automatically test things like this? * ''By decoupling! You create the bevavior you want OUTSIDE the UI, and then use the UI to render it. This often means that you don't make use of convenient facilities in the UI. Instead you create presentation models that behave like the UI, but that do not contain any UI code. And finally you wire those presentation models to the UI using very simple glue code. == UB'' *! !* Wed, 19 Oct 2005 14:36:37, Dave Bouwman, UI Testing... We've got a little bit of a problem with UI testing, and heres a quick run down of why. We develop customizations for the ESRI ArcGIS system (www.esri.com for more on that). Since GIS is very very heavy on UI interactions, and we're building on someone elses framework and/or application, we're at a loss for how to test it. We've looked at test complete and other "snapshot" type systems, and they're not worth implementing for the scale of projects we do (month or two at a time). We try to decouple as much as possible, but since we rely on ESRI's underlying objects, and interface objects, there are many cases where we can not decouple. Ideas? Dave *! !* Mon, 21 Nov 2005 03:14:31, Carlos C Tapang, What if the next logical test requires more than 10 minutes of coding? Has there been a response to Keith Gregory's question above? If there has, I am missing it. I suppose the answer is to write tests for the inner class also? *! !* Tue, 22 Nov 2005 14:19:20, John D. Mitchell, Skip lists and TDD Keith, In terms of testing, your existing 'unit' tests are basically just API-level functional tests. If you want to make meta-data (e.g., statistics) about the data structure part of your API, that's up to you. However, given your aims for the module, it sounds like you really need some 'white-box'/internal/private tests. In terms of TDD, I don't see how you could have possibly implemented that inner class and those private methods by following e.g., the 'rules' of TDD. I.e., how did you have failing tests each step of the way that lead you to all of that (internal) functionality? It sounds like you must have taken some awfully larger steps in a premeditated direction in your coding. *! !* Tue, 22 Nov 2005 17:24:52, Uncle Bob, What if the next logical test requires more than 10 minutes of coding? |Carlos C Tapang asked: ''Has there been a response to Keith Gregory's question above? If there has, I am missing it.''| First, I ''would'' start with a simple linear list implementation. The reason I would do that is that it would prove that my tests are correct. Any tests that pass for a linear list, shoudl pass for a skip list. Eventually, in order to get the Skip List implementation, I'd have to write some tests that a linear list would fail, and that only a skip list would pass. I would choose very tiny incremental test cases so that I did not have to implement the whole Skip List in one great leap. Choosing these tiny incremental tests is part of the ''art'' of TDD. Finally, I think you should lose the attitude: "Just for the tests." I don't mind exposing a variable, function, or inner class if it helps me write the tests. I don't find that I have to do it too often; but I don't mind doing it. Also the extra 'distribution' function doesn't bother me. Again, I don't find that I have to do things like that very often, but the tests are so important that I certainly will expose functions, or add utilities, if it supports the tests. *! !* Mon, 28 Nov 2005 17:52:47, Keith Gregory, What if the next logical test requires more than 10 minutes of coding? |Uncle Bob writes: Finally, I think you should lose the attitude: "Just for the tests." I don't mind exposing a variable, function, or inner class if it helps me write the tests. Perhaps. My concern, however, is that breaking encapsulation is a slippery slope. Today it's just for the tests, tomorrow it's because another class becomes simpler when it has access to the internals of the first. I've met a bunch of smart programmers who slid down that slope, and am trying not to join their ranks. To that end, I came up with three alternative approaches: First is to create a "statistics" object that's updated by the internal methods of SkipList and read by the tests. Not quite a mock object, but same idea. The primary risk here is collecting the wrong statistics. I also think that putting a lot of work into the mock disrupts the flow of small increments. Second, implement "in vitro" and move the tested code into the final class. You end up with "published" tests for the API alone, but development proceeds in small increments. Third, extract the core of the list into its own class, and test that while you're developing it. Probably the cleanest approach, but it seems that unnecessary ''decoupling'' is a smell -- it goes against "do the simplest thing that can possibly work." |Choosing these tiny incremental tests is part of the ''art'' of TDD. That's true, and I'd like to see some exploration of that art in a realistic setting. The Bowling Game is a nice way to demonstrate TDD on a micro scale, but let's face it, it works because it has a limited story: "score a bowling game after all the balls have been rolled." (btw, how do you avoid an IndexOutOfBounds in testPerfectGame()?) *! !* Fri, 2 Dec 2005 15:03:44, John Cowan, Bah Type the first character of a test, compile it; it fails, naturally. Type the next character of a test, compile it; it fails, naturally. Repeat. We keep rules 1, 2, and 3, but we never deliver anything. *! !* Fri, 2 Dec 2005 16:15:56, Bill Krueger, Managability of 1000's of test cases With the pro's come the con's. In this case, there's having to deal with the potentially overwhelming number (1000's) of test cases. For the reasons they you propose they have worth for examples and documentation, the numbers seem to negate that. That said, I do like your thoughts. *! !* Sat, 3 Dec 2005 19:21:37, Uncle Bob, Bah: We never deliver anything. |John Cowan said:''Type the first character of a test, compile it; it fails, naturally. Type the next character of a test, compile it; it fails, naturally. Repeat. We keep rules 1, 2, and 3, but we never deliver anything.''| John, Perhaps you are taking this too literally. There are many examples available for how to apply these laws. Consider, for example, TheBowlingGameKata. *! !* Sat, 3 Dec 2005 19:28:56, Uncle Bob, 1000s of test cases. Yes, there can be thousands of test cases. The more production code you have, the more test cases you will have. Fortunately, the structure of the tests mimics the structure of the production code. For every class, there is often a corresponding test-class. This test-class describes how the class works, how to call it, and what to expect from it. Personally, I keep the test classes in the same package as the production code class. So when I look in my IDE I see the class, and the test-class right next to each other. That's ''very'' nice. *! !* Mon, 5 Dec 2005 11:38:17, Matt Segvich, Re: Managability of 1000's of test cases |Bill Krueger said:''With the pro's come the con's. In this case, there's having to deal with the potentially overwhelming number (1000's) of test cases.''| From my experience, I've never had to deal with all the test cases at once, rather just the ones that are impacted by the changes. Which for me is typically a handful. The only time I've seen others struggle is when they put off testing instead of doing it in-step with the changes. The longer you put them off, the more tests you'll need to update. Also remember, to find the ones that you've broke you just run the tests. For me the tests allow me to sleep at night. In fact, more often than not I find (due to bugs) I have too few tests. *!
Use alt+s (Windows) or control+s (Mac OS X) to save your changes. Or, tab from the text area to the "Save" button!
Grab the lower-right corner of the text area to increase its size (works with some browsers).