Architecture Posts

Ideas, Experiences & thoughts worth Sharing!!!

Monthly Archives: July 2014

Behavior Driven Development (BDD) – Best Practices

BDD is a relatively new method for developing software, which is essentially a refined version of test driven development (TDD). Where the TDD uses more traditional tools in developing the tests (QTP, etc), BDD introduces a new paradigm of testing framework & tools. In BDD the tests are written using a language called Gherkin, which follows a high level “Given … When…. Then..” format to define the test (requirement) criteria.

This framework is widely being adopted now and there are tools support in java, .Net and open source technologies. While I had my reservations initially in its practicality, after using it for a few weeks (in a real world financial domain project), it became clear that there is actually value in adopting BDD.

One of striking aspect was the discipline required for it to be effective and re-usable in large enterprises. The discipline comes from well-established best practices and a governance model (ensure compliance) to support the framework & tools. This post doesn’t go into the technical framework and tools which I found mostly covered in a number of blogs. I intend to cover some of the best practices we have created and followed to make this as beneficial as it can be.

Best Practices

Best practices are guidelines that are applicable at a certain time point and can change over a period of time depending on the organization strategy, project context, etc. This section should list the current active best practices at any point of time.

Test pyramid

Description

BDD supports automating component & UI testing. And as a general good practice for maintenance and future enhancement it is recommended that the test pyramid approach is followed in terms of number of tests and the coverage across the spectrum of automated tests. (Unit, Component (Api) and UI tests).

 

Rational

http://martinfowler.com/bliki/TestPyramid.html

And other numerous web resources.

Example Not applicable.

Test Background

Description

Test background should be limited only to any pre-requisite activity that doesn’t have any individual test specific action or data.

These would activities such as logging in, cleaning up or setting up base data (Not specific to individual tests).

 

Rational

Having test specific data setup likely to change for individual test cases, resulting in complex background feature statements.

 

Example

Consider a feature (test pack) requiring keying in screens 1 to 5 with 5 difference combination of data (scenarios).

If for all scenarios screens 1-3 remains identical, there would be an urge to move this statement to the background of the feature, hence resulting in less duplication.

But this is not recommended as even though the steps appear identical they may be processing different data set and in future one of these screens might need different action for some of the scenarios.   This will result in rework of the entire feature rather than just modifying the impacted scenarios.

The duplication could be avoided by abstracting at the right level. I.e. instead of having statement to separately key screen 1, screen 2 and screen 3, there could one generic feature statement to enter these details all in one feature statement.

Test Objectives

Description

Establishing the test objectives is the key. In principle each scenario in a feature should cover a single objective. However this need not be strict rule, if it would be more suitable for specific scenarios to cover multiple objectives. (Validation & results for example).

As a guideline UI tests should only cover end to end happy paths. (Warning messages could be included as applicable).

Test objectives not to be confused with test conditions. Each objective can have multiple test conditions (verification points).

 

Also there is a general myth that each scenario should only have one Then statement(s). There is no valid reason not to have more than one Then statement.   I.e. a scenario could have When <do something> Then <test something> When <do something more> Then <test something else>. While it’s true that we want to keep scenarios small and manageable, having just one then statement as a restriction is not one of the best practices.   It just increases the number of scenarios and increased maintenance overhead.

Rational Separation of concern and easier maintenance.
Example

For a scenario where keying details with missing information and submitting to continue, there are potentially two objectives,

a)       Test the missing information warnings are appearing as expected

b)       Test the submitted details are saved and confirmation displayed.

As the warning is part of the same end to end process where no additional actions are required, it could be included within the same scenario.

However if system required to key in additional details and click separate buttons that are only required for testing the warning and not so for completing the end to end flow, it will be good practice to keep these in two different scenarios.

Process reuse

Description

A library of actions at individual component or UI object level is maintained in order not to repeat the same.

The library should provide the feature statement and details of corresponding actions. The SpecFlow framework (Visual Studio) already provides a natural hierarchy but delivery process must ensure the compliance.

Rational Prevents TA/BAs from writing the same actions in different formats. Since the BDD feature file is written in English like language (Gherkin), same actions could be written in a number of different ways
Example

Must avoid scenarios as below, where product details are entered in multiple feature statements; however the underlying action remains the same.

Scenario: One

When I enter Product details

And I Enter Contact details

...

 

Scenario: Two

When I Enter Contact and Product Details click on Continue button

...

 

Scenario: Three

When I enter spare parts details

...

In order to prevent these, a library becomes essential. Visual Studio does provide a lookup like option to support it, but having this library pre-defined would help better management of common actions.

Data driven (inline vs external)

Description

Ensure the feature scenarios are data driven as much as possible. Whenever multiple scenarios are varying only by the data being entered or the validation conditions, it must be designed as data driven scenario.

There are two approaches used for driving the tests using data.

a)       External Data as csv (s/sheet) – Are to be used when large amount of input data to be keyed in for test scenarios (like keying product details, etc)

b)       Inline data tables (as part of Gherkin language) – Recommended for verifying data driven expected results, or minimal data keying.

Rational

Avoids duplication of scenarios and allows greater flexibility for maintenance.

 

Example

Instead of below,

@Carburetor

Scenario: Add Spare parts

When I enter product details manually and navigate to results page

Then Verify that Product type is SpareParts

 

@Rice

Scenario: Add Grocery

When I enter product details manually and navigate to results page

Then Verify that Product type is Grocery

 

@Carpet

Scenario Outline: Add Interior decorator test

When I enter product details manually and navigate to results page

Then Verify that product type is Interior Decorator

Use data driven approach as below,

@smoke @AddWorker

Scenario Outline: Add Product Manually

When I enter product details manually and navigate to results page

Then Verify that Product type is '<Product_Type>'

 

@Carburetor

Examples: Add Spare parts

| Product_Type|

| Spare Parts |

@Rice

Examples: Add Grocery

| Product_Type   |

| Grocery |

 

@Carpet

Examples: Add Interior decorator test

| Product_Type           |

| Interior Decorator |

This would result in common definition file and page objects hence less maintenance. Also additional tests could be added with ease.

 

Principles

Principles are best practices that are more generic than the best practices. These are applicable across the entire organization and all the time without any time bounds, while the best practices, could be specific to some division and may be applicable only during a certain time period. Best practices also evolve over the time whilst principles are fairly stable.

Independent

Description Scripts should be independent of other scripts, so that these could be run on their own.
Rational As the test pack keeps growing, it is important to be able to investigate individual failures & be able to enhance for the future changes. If a complex inter-dependency is created, it would become more difficult and may end up unmanageable for newer project teams.

Idempotent

Description Test pack or individual script should be re-runnable on a target environment and resulting in exact same behaviour as the first run.
Rational It would require running a test pack multiple times in an environment. And having to undo (clear) data updates would prove unmanageable in the longer run. Hence the script should either be designed to clean up its updates, or ensure existence of required state of data as part of background to carry out the test. This will ensure user who may be unfamiliar with a particular functionality is still able to run and analyse the results.
Example Not applicable

Business language

Description The Feature file should be written in business language without reference to technical objects.
Rational The ultimate objective of BDD is to have business understandable specifications are actually used for validating the system built. When these specifications become technical, it defeats the whole purpose of Behaviour Driven test pack.
Example

The following Add Product scenario is entering product details manually by selecting the particular option. The incorrect way of doing this would be,

@AddProduct

Scenario: Add product manually

When I Click on Add product image on Admin dashboard

And  I Click on Add product button in Add individually radio option

Then Product Details page is displayed

When Product details and click on Continue button

In this particular instance, reference to the browser page elements like image, button and actions like click are very low level and technical. It makes the understanding complex and also increases maintenance as these details could change over the application lifecycle. Hence this could be written as below.

@AddProduct

Scenario: Add product manually

When I select Add product individually option

Then Product details form is presented

When I enter product details and submit

This is more open and not referring any technical objects. The abstraction level is also depends very much on the objective of the test. In this particular instance since the objective is to test the validation, these steps are scripted individually. If the objective is just to test the end to end process, the keying in process could be written in a single feature statement as required.

 

 

ploeh blog

Ideas, Experiences & thoughts worth Sharing!!!

Passionate about data

Ideas, Experiences & thoughts worth Sharing!!!

You’ve Been Haacked

Ideas, Experiences & thoughts worth Sharing!!!

Swaroop C H - India, Startup, Technology, Life Skills

Conning people into thinking I'm intelligent. Since 1982.

Paul Graham: Essays

Ideas, Experiences & thoughts worth Sharing!!!

Martin Fowler

Ideas, Experiences & thoughts worth Sharing!!!

Scott Hanselman's Blog

Ideas, Experiences & thoughts worth Sharing!!!