FitNesse.
UserGuide.
FitLibraryUserGuide.
SuiteFixtures.
SuiteFixtureDetailsAndRationale [add child]
SuiteFixtureDetailsAndRationale [add child]
With suite fixtures:
We now look at the rationale for each of these capabilities. Please note that the description that follows assumes an understanding of the execution model of Fit. I plan to include a customer-friendly description later.
As we have argued elsewhere, a more powerful approach is to express the storytests once in terms of the business domain. The same storytests can then be run with different fixtures so that the testing can be carried out at different levels.
The simplest approach to handling this is to change the fixture class names in the first table of each of the storytests. But this is a boring and error-prone approach. Of course, the change process could be automated, but there are better ways.
Another approach is to have distinct sets of fixtures with the same names, which are switched in according to the classpath in Java that is being used (or the equivalent in other languages). But this can be confusing and makes it difficult to share code between distinct sets of fixtures.
The approach we take is to remove fixture class names entirely from storytests (just as DoFixture made it possible to eliminate all fixture class names except in the first table). Instead, a table near the start of the storytest identifies the fixture class indirectly, with a name that is unrelated to the fixture class name. The suite fixture object interprets the tables in the storytest until it carries out an action that results in a DoFixture object. It then passes repsonsibility for interpreting the rest of the storytest to that fixture object.
This is similar to the approach already used in DoFixture. (Actually, SuiteFixture is a subclass of DoFixture, so there is little extra mechanism to permit this.)
So, in general, it's not possible to organise the storytests into a single hierarchy to serve all purpose.
Now FitNesse[?] permits symbolic links and so multiple overlapping suites can be defined. However, a separate suite needs to be defined for each of the combinations of use. Once the number of combinations grow, the suites become harder to manage.
The approach we take with suite fixtures is to allow for each storytest to be classified as being in multiple categories, through the use of keyword. For example, all storytests that are completed can have the keyword "completed", and the build machine only selects those ones.
The filtering is carried out by having a table of keywords at the start of the storytest. The suite fixture interprets this table and determines whether to continue running this storytest, based on the keywords.
A default approach is provided for doing this filtering, but it is very simple to extend or alter the filtering mechanism, as it is based on a DoFixture approach to table intepretation. For example, one company has keywords for the subsystems, such as "a.b.c". Their filtering mechanism takes account of this naming convention, so that if the selected keywords include "a", or "a.b", or "a.b.c", then a storytest with a keyword of "a.b.c" will be selected and run.
As well as associating a list of keywords with a storytest, a mechanism is needed to specify which keywords are used for selection when a suite is run. The mechanism of specifying these selected keywords differs between FitNesse and FolderRunner, as we discuss below.
We may like a suite of tests to make use of a resource that's expensive (or annoying) to acquire afresh with each storytest, such as a database connection or Spring configuration. Of course, the resource can't be changed in important ways between the storytests because we want to retain test independence.
Usually, each storytest is started with a fresh fixture object. To share a resource that's already allocated, that fixture object needs to explicitly access the resource through a static (class) variable.
Instead, such a resource can be created by the suite fixture and shared between the fixture objects. As it is the responsibility of the suite fixture to create the fixture object for each storytest, it can pass any such resources as parameters to the created fixture object. This is very similar to the way in which the first DoFixture object for a storytest is responsible for creating the fixture objects for subsequent tables in that storytest.
- Fit pages/files (storytests) no longer need to mention fixture class names. This means that the same storytests can easily be used with different fixtures for testing at different levels, such as directly into the domain layer and through a GUI or web interface.
- Storytests can be filtered for a particular test run. For example, when only the completed storytests should be run on the build machine.
- The fixtures for the storytests in a suite can easily share resources, such as database connections.
- Each suite can provide different configuration information, such as selecting a DB or Spring configurations
We now look at the rationale for each of these capabilities. Please note that the description that follows assumes an understanding of the execution model of Fit. I plan to include a customer-friendly description later.
Multi-purpose Storytests
There is a temptation to write different storytests when testing different levels of a system. For example, when explicitly testing through the UI, the storytests will be expressed very differently than when explicitly testing through the domain layer or directly into a subsystem. But this leads to redundancy between the storytests. In addition, the storytests for the UI will be verbose, it will hard to see the essence of the domain in those storytests, and they will be hard to change.As we have argued elsewhere, a more powerful approach is to express the storytests once in terms of the business domain. The same storytests can then be run with different fixtures so that the testing can be carried out at different levels.
The simplest approach to handling this is to change the fixture class names in the first table of each of the storytests. But this is a boring and error-prone approach. Of course, the change process could be automated, but there are better ways.
Another approach is to have distinct sets of fixtures with the same names, which are switched in according to the classpath in Java that is being used (or the equivalent in other languages). But this can be confusing and makes it difficult to share code between distinct sets of fixtures.
The approach we take is to remove fixture class names entirely from storytests (just as DoFixture made it possible to eliminate all fixture class names except in the first table). Instead, a table near the start of the storytest identifies the fixture class indirectly, with a name that is unrelated to the fixture class name. The suite fixture object interprets the tables in the storytest until it carries out an action that results in a DoFixture object. It then passes repsonsibility for interpreting the rest of the storytest to that fixture object.
This is similar to the approach already used in DoFixture. (Actually, SuiteFixture is a subclass of DoFixture, so there is little extra mechanism to permit this.)
Filtering Storytests
Often, you don't want to run all of the storytests in a suite:- Only completed storytests should be run on the build machine, as storytests that are in development progress will fail. However, it's a pain to have to organise suites around whether storytests are completed or not. It makes much more sense to organise the storytests around the modules and etc of the domain model.
- When altering a part of a large system, faster progress can be made by running more often those storytests that are most relevant. However, some changes will have an impact on several parts of the domain. Likewise, some storytests will impact on several parts, such as related to two Entities.
- When testing through the UI, which is slow, only some storytests may be used.
So, in general, it's not possible to organise the storytests into a single hierarchy to serve all purpose.
Now FitNesse[?] permits symbolic links and so multiple overlapping suites can be defined. However, a separate suite needs to be defined for each of the combinations of use. Once the number of combinations grow, the suites become harder to manage.
The approach we take with suite fixtures is to allow for each storytest to be classified as being in multiple categories, through the use of keyword. For example, all storytests that are completed can have the keyword "completed", and the build machine only selects those ones.
The filtering is carried out by having a table of keywords at the start of the storytest. The suite fixture interprets this table and determines whether to continue running this storytest, based on the keywords.
A default approach is provided for doing this filtering, but it is very simple to extend or alter the filtering mechanism, as it is based on a DoFixture approach to table intepretation. For example, one company has keywords for the subsystems, such as "a.b.c". Their filtering mechanism takes account of this naming convention, so that if the selected keywords include "a", or "a.b", or "a.b.c", then a storytest with a keyword of "a.b.c" will be selected and run.
As well as associating a list of keywords with a storytest, a mechanism is needed to specify which keywords are used for selection when a suite is run. The mechanism of specifying these selected keywords differs between FitNesse and FolderRunner, as we discuss below.
Shared Resources
Suite fixtures allow for the sharing of resources between all the fixtures for the storytests in a suite.We may like a suite of tests to make use of a resource that's expensive (or annoying) to acquire afresh with each storytest, such as a database connection or Spring configuration. Of course, the resource can't be changed in important ways between the storytests because we want to retain test independence.
Usually, each storytest is started with a fresh fixture object. To share a resource that's already allocated, that fixture object needs to explicitly access the resource through a static (class) variable.
Instead, such a resource can be created by the suite fixture and shared between the fixture objects. As it is the responsibility of the suite fixture to create the fixture object for each storytest, it can pass any such resources as parameters to the created fixture object. This is very similar to the way in which the first DoFixture object for a storytest is responsible for creating the fixture objects for subsequent tables in that storytest.
Add Child Page to SuiteFixtureDetailsAndRationale