ArticleS. PaulPagel.
SententialTesting [add child]

Sentential Testing.

In a project I have been working on, we recently decided to allow wrapped assertions to customize our unit tests. Our purpose was to make the unit tests be sentential enough to explain the meaning of the assertion in the execution of the assertion.

We can’t get rid of all developer comments (variable and method names) because, if we did, we would lose the ability to easily read and, consequently, easily maintain the code. In fact, writing easily readable code is a primary goal of good practices, since it is directly correlated with making code easily maintainable. This is important because maintenance is the most expensive part of the development process. However, we must be cautious about writting comments, since they can often have the opposite effect—less maintainable code—when they are misrepresentations or plain lies. There is a balance that results in easily read and understood tests. Investing in writing tests to tell a story to the developers who may have to look at the tests in the future, makes maintenance and TDD easier and with a better flow.

We chose to achieve this balance by merging the assertions with the comments of the execution of the test, resulting in a pseudo-sentence. The statement is an explanation for the next developer to read. The form we chose is


An example of one of these assertions is

public void test_PressingBrakesShouldDecelerateVehicle()

int originalSpeed = vehicle.currentSpeed;

Verify.That(“Brakes decelerate vehicle.”).ProvedBy(vehicle.currentSpeed).IsLessThan(originalSpeed);

When a developer comes across this story, along with “PressingEmergencyBrakeLocksTires(), SlamOnBrakesShouldSkid(), and BrakesShouldAntiLockAtIntervals(),” the developer is told a story of what the class is doing. Why do we use custom asserts when good test names will solve the problem?

Changing tests should be cheap. As new designs and behavior evolve in the system, tests are changed to conform to the new structures. When implementing a new design to an existing system (such as breaking up an inheritance in favor of composition), I change the tests to fit the new structure on a macro level. However, this often implies that I am changing the behavior (if only slightly) to please the new design. Yet, test names, like comments, are not executable, so you can’t trust them to tell you what the tests test. On a system with heavy refactorings, a test might have its details changed five times without the developer thinking to verify the integrity of the names. An answer is to write sentential explanation of the test as the assert statement since the assert statement executes. I will trust execution over comments every time.

There is a project in its early stages called NSpec(.NET), which is a framework for assertions of a more sentential form. The project is based on the idea of Behavior Driven Development (BDD). RSpec, the same project for ruby has a form of BDD assertions which take full advantage of the features of the dynamic language. It allows you to add the assertion to the object from the test.

For example:

def pressing_brakes_should_decelerate_vehicle

This is really cool! It defines the test as behavior of the object, which it is. However, when the code isn’t ruby, there needs to be other means in place to bind the description of the assertion to something which executes.

 Fri, 20 Jan 2006 05:37:12, nraynaud, Tests in a changing world
You're right, we are slowly going to agile world, and for now we have lots of meaningless test taken randomly from existing code (does it makes sense putting a multi megabytes source file in the compiler you test and expecting exactly 2008 errors ? without even covering 10% of the compiler). We work on a language, yesterday, I added a restriction to the langugage, it took me 10 minutes to implement and test the change but the rest of the day to change all the others tests.

Moreover, I thank Intellij Idea that renames not only classes but also varialbes named after that class and even comments, this is on the way you takes.
 Fri, 20 Jan 2006 13:13:22, David Chelimsky, should_be_decelerating
I've just submitted a contribution to rspec that will make this even cooler. If vehicle can respond to any of these messages with a boolean value:

then you can write this in your test:
def pressing_brakes_should_decelerate_vehicle

Admittedly, this may be confusing to a programmer who is looking for a "should_be_decelerating" method on vehicle, but the messaging is clear it fails ("<is_decelerating?> should be true") and it definitely improves the communication of intent. If you're doing BDD (i.e. spec-first), you'll get a NoMethodError on "decelerating". Then it's up to you to implement a method responding to any of the forms described above.
 Fri, 20 Jan 2006 13:57:32, ,
How about using Higher Order Messaging style and let the user write:


The test framework could mix should_be and similar higher-order messages into the Kernel module.
 Fri, 20 Jan 2006 14:40:41, David Chelimsky, should_be
That's kind of how rspec works now. should_equal, should_contain, should_be_true, etc are all mixed in to Object, so you can do things like some_object.some_attribute.should_be_true. Your suggestion of tying it to a "should_be" method (rather than parsing the should_be_xxx message) might be more clear. I think, though, that "should_be.decelerating?" is weird to read (because of the question mark). Programatically, however, it does make the statement look more like the implementation. I'll think on it some more...
 Sun, 6 Aug 2006 11:29:33, leoyang22, ok!