Ideas, Experiences & thoughts worth Sharing!!!
Behavior Driven Development (BDD) – Best Practices
BDD is a relatively new method for developing software, which is essentially a refined version of test driven development (TDD). Where the TDD uses more traditional tools in developing the tests (QTP, etc), BDD introduces a new paradigm of testing framework & tools. In BDD the tests are written using a language called Gherkin, which follows a high level “Given … When…. Then..” format to define the test (requirement) criteria.
This framework is widely being adopted now and there are tools support in java, .Net and open source technologies. While I had my reservations initially in its practicality, after using it for a few weeks (in a real world financial domain project), it became clear that there is actually value in adopting BDD.
One of striking aspect was the discipline required for it to be effective and re-usable in large enterprises. The discipline comes from well-established best practices and a governance model (ensure compliance) to support the framework & tools. This post doesn’t go into the technical framework and tools which I found mostly covered in a number of blogs. I intend to cover some of the best practices we have created and followed to make this as beneficial as it can be.
Best practices are guidelines that are applicable at a certain time point and can change over a period of time depending on the organization strategy, project context, etc. This section should list the current active best practices at any point of time.
BDD supports automating component & UI testing. And as a general good practice for maintenance and future enhancement it is recommended that the test pyramid approach is followed in terms of number of tests and the coverage across the spectrum of automated tests. (Unit, Component (Api) and UI tests).
And other numerous web resources.
Test background should be limited only to any pre-requisite activity that doesn’t have any individual test specific action or data.
These would activities such as logging in, cleaning up or setting up base data (Not specific to individual tests).
Having test specific data setup likely to change for individual test cases, resulting in complex background feature statements.
Consider a feature (test pack) requiring keying in screens 1 to 5 with 5 difference combination of data (scenarios).
If for all scenarios screens 1-3 remains identical, there would be an urge to move this statement to the background of the feature, hence resulting in less duplication.
But this is not recommended as even though the steps appear identical they may be processing different data set and in future one of these screens might need different action for some of the scenarios. This will result in rework of the entire feature rather than just modifying the impacted scenarios.
The duplication could be avoided by abstracting at the right level. I.e. instead of having statement to separately key screen 1, screen 2 and screen 3, there could one generic feature statement to enter these details all in one feature statement.
Establishing the test objectives is the key. In principle each scenario in a feature should cover a single objective. However this need not be strict rule, if it would be more suitable for specific scenarios to cover multiple objectives. (Validation & results for example).
As a guideline UI tests should only cover end to end happy paths. (Warning messages could be included as applicable).
Test objectives not to be confused with test conditions. Each objective can have multiple test conditions (verification points).
Also there is a general myth that each scenario should only have one Then statement(s). There is no valid reason not to have more than one Then statement. I.e. a scenario could have When <do something> Then <test something> When <do something more> Then <test something else>. While it’s true that we want to keep scenarios small and manageable, having just one then statement as a restriction is not one of the best practices. It just increases the number of scenarios and increased maintenance overhead.
|Rational||Separation of concern and easier maintenance.|
For a scenario where keying details with missing information and submitting to continue, there are potentially two objectives,
a) Test the missing information warnings are appearing as expected
b) Test the submitted details are saved and confirmation displayed.
As the warning is part of the same end to end process where no additional actions are required, it could be included within the same scenario.
However if system required to key in additional details and click separate buttons that are only required for testing the warning and not so for completing the end to end flow, it will be good practice to keep these in two different scenarios.
A library of actions at individual component or UI object level is maintained in order not to repeat the same.
The library should provide the feature statement and details of corresponding actions. The SpecFlow framework (Visual Studio) already provides a natural hierarchy but delivery process must ensure the compliance.
|Rational||Prevents TA/BAs from writing the same actions in different formats. Since the BDD feature file is written in English like language (Gherkin), same actions could be written in a number of different ways|
Must avoid scenarios as below, where product details are entered in multiple feature statements; however the underlying action remains the same.
Scenario: One When I enter Product details And I Enter Contact details ... Scenario: Two When I Enter Contact and Product Details click on Continue button ... Scenario: Three When I enter spare parts details ...
In order to prevent these, a library becomes essential. Visual Studio does provide a lookup like option to support it, but having this library pre-defined would help better management of common actions.
Ensure the feature scenarios are data driven as much as possible. Whenever multiple scenarios are varying only by the data being entered or the validation conditions, it must be designed as data driven scenario.
There are two approaches used for driving the tests using data.
a) External Data as csv (s/sheet) – Are to be used when large amount of input data to be keyed in for test scenarios (like keying product details, etc)
b) Inline data tables (as part of Gherkin language) – Recommended for verifying data driven expected results, or minimal data keying.
Avoids duplication of scenarios and allows greater flexibility for maintenance.
Instead of below,
@Carburetor Scenario: Add Spare parts When I enter product details manually and navigate to results page Then Verify that Product type is SpareParts @Rice Scenario: Add Grocery When I enter product details manually and navigate to results page Then Verify that Product type is Grocery @Carpet Scenario Outline: Add Interior decorator test When I enter product details manually and navigate to results page Then Verify that product type is Interior Decorator
Use data driven approach as below,
@smoke @AddWorker Scenario Outline: Add Product Manually When I enter product details manually and navigate to results page Then Verify that Product type is '<Product_Type>' @Carburetor Examples: Add Spare parts | Product_Type| | Spare Parts | @Rice Examples: Add Grocery | Product_Type | | Grocery | @Carpet Examples: Add Interior decorator test | Product_Type | | Interior Decorator |
This would result in common definition file and page objects hence less maintenance. Also additional tests could be added with ease.
Principles are best practices that are more generic than the best practices. These are applicable across the entire organization and all the time without any time bounds, while the best practices, could be specific to some division and may be applicable only during a certain time period. Best practices also evolve over the time whilst principles are fairly stable.
|Description||Scripts should be independent of other scripts, so that these could be run on their own.|
|Rational||As the test pack keeps growing, it is important to be able to investigate individual failures & be able to enhance for the future changes. If a complex inter-dependency is created, it would become more difficult and may end up unmanageable for newer project teams.|
|Description||Test pack or individual script should be re-runnable on a target environment and resulting in exact same behaviour as the first run.|
|Rational||It would require running a test pack multiple times in an environment. And having to undo (clear) data updates would prove unmanageable in the longer run. Hence the script should either be designed to clean up its updates, or ensure existence of required state of data as part of background to carry out the test. This will ensure user who may be unfamiliar with a particular functionality is still able to run and analyse the results.|
|Description||The Feature file should be written in business language without reference to technical objects.|
|Rational||The ultimate objective of BDD is to have business understandable specifications are actually used for validating the system built. When these specifications become technical, it defeats the whole purpose of Behaviour Driven test pack.|
The following Add Product scenario is entering product details manually by selecting the particular option. The incorrect way of doing this would be,
@AddProduct Scenario: Add product manually When I Click on Add product image on Admin dashboard And I Click on Add product button in Add individually radio option Then Product Details page is displayed When Product details and click on Continue button
In this particular instance, reference to the browser page elements like image, button and actions like click are very low level and technical. It makes the understanding complex and also increases maintenance as these details could change over the application lifecycle. Hence this could be written as below.
@AddProduct Scenario: Add product manually When I select Add product individually option Then Product details form is presented When I enter product details and submit
This is more open and not referring any technical objects. The abstraction level is also depends very much on the objective of the test. In this particular instance since the objective is to test the validation, these steps are scripted individually. If the objective is just to test the end to end process, the keying in process could be written in a single feature statement as required.