Some days ago I was thinking about the good practices of testing. Immediately after the first commandment “test code is First Class code” it comes the “Run test code really often”.
If you run the test code often, you immediately know when you have broke something. Think to the following scenario, you write 200 lines of new code, then you decide to write tests for them, but you discovered that a large amount of old tests are broken; now it is difficult to say where did you introduce bugs, and you need to spent time to find the cause of the error.
Now think to a little different scenario, you keep nunit open, you configure nunit to: automatically reload test assembly and run test again. Then you write some lines of code, then press CTRL+SHIFT+B, your code gets compiled and nunit immediately runs the tests. Now imagine that you compile often, you change very few lines of code, compile and… whenever you see red, you really have to inspect only few lines to understand where and why the code is broken.
At the other end of the spectrum you find situation like the following. Developer A had finished a 5 hours session where he implemented a lot of new features for the project, now he run old tests but a lot of them failed. After 15 minutes most of the tests are working again but there are still some of them that are red…. and the developer really do not understand why, after all he really wrote a lot of new code. Since he want to test the new features, the old tests that fail are simply commented out or made [Explicit] with the excuse “I’ll fix them later”….with the risk that noone will fix them again.
This lead to the original post, you always need to take care of test code, make sure that he has good quality, run tests as often as you can, and refactor them often to remove fragility.
alk.
Tags: Unit Testing
July 11th, 2008 at 4:11 pm
When I first tried to force myself to religiously do the following, but now if I skip a step it makes me terribly uncomfortable (except in certain very specific situations where it is best to do so.):
1. svn up.
2. rebuild.
3. Run all tests, making sure they pass.
4. Write a new test or tests (or modify an existing one or ones) that will code the code I am about to add (or change).
5. Run all tests, making sure the step 4 tests fail.
6. Write the code to cover one of the failing tests.
7. Run all tests, making sure the step 6 test no passes.
8. Repeat 6 and 7 until their tests all pass.
9. Optionally go back to step 1.
10. svn up.
11. rebuild.
12. Run all tests, making sure they pass.
10. svn commit.
If there is any part of that list that people fail to execute because they fail to understand it, it is around steps 4 and 5. It is critical that the tests be written and/or changed BEFORE code is changed, and that they fail. It is only by starting off with them failing that you can know for certain that your changes are what made them pass, as opposed to writing tests that pass and just going on faith that they actually cover your changes.
The only time I now change steps 4 and 5 is on the infrequent occasion that I am removing known-dead but still-tested code. Every other time, I update the tests first, make them fail, and then code to make them pass.
July 11th, 2008 at 4:16 pm
Ouch. Typo city.
“When I first tried to force myself to religiously do the following”… it often felt weird and was difficult to get used to …”, but now if I skip a step it makes me terribly uncomfortable”
“4. Write the test or tests” … “that will” _cover_ “the code I am about to add (or change).
“7. Run all tests, making sure the step 6 test _now_ passes.”
“1_3_. svn commit.” (how can you tell I inserted the final svn up, rebuild, run all tests set before the final commit
How can you tell that it’s Friday and it is time for me to go home from work and rest up after a crazy week.
July 12th, 2008 at 6:14 am
You described the TDD way to build a software, you also pointed out a very good practice, commit often.
. The bad thing is that seems difficult to bring this way of writing software to project manager… I often heard the horrible
“We are short of time, we cannot waste time on test”
Writing no test is surely not a way to speed up the project, absolutely!!
alk.