Architecture Posts

Ideas, Experiences & thoughts worth Sharing!!!

Category Archives: open2Test

Open 2 Test Vs Telerik Studio

Recently I had to do a test tool evaluation in trying to establish the right approach for a client. This involved comparing the Open 2 Test framework using Selenium web Driver and the Telerik test suite which the client already had in place for some of their projects. The objective is to aid establishing the key drivers and customizing a solution that achieves the maximum benefit for the investment. This is specifically done for testing web applications, hence the focus is on web functional testing, and however I have also listed other aspects separately below.

Web functional testing

 

Attribute

Open 2 Test

Selenium Web Driver

Telerik Test Studio

Comments

Cost

Free. Open source software based. Uses common office software like MS Excel for scripting tests.

Commercial product licensed per duration. (To check if it supports unlimited users?)

Well open source is not all free, and will require an initial investment to extend and customize the tool as required. So this need to be considered in the right context.

Skills & Learning Requirement

Predefined set of simple keywords used for scripting.

UI based scripting language.

Record & Play mechanism.

Both are intuitive to large extent and may require some amount of learning & basic training.

Integration with TFS

Extensible. Java based, hence could leverage on numerous open source libraries available.

Built-in support

While Telerik supports full integration with Visual Studio & TFS, O2T could be extended & customized to integrate as per the specific requirement. (Logging defects in a specific TFS project, etc)

Multi browser support

Existing support for IE & FF. Could be extended for others, but will require framework enhancement effort.

IDE support only for FF

Built-in support for all major browsers for both recording (scripting) and testing.

Telerik’s feature enables cross browser testing automatically.

Test pack maintainability

Fully hand-crafted scripting, hence relatively easier to update & maintain.

Decouples the page DOM elements using object repository, hence easier to maintain page element changes.

Supports script re-usability.

Recorded scripts hence will require effort to update for maintenance.

Allows in-line script editing.

Allows grouping of tests into a re-usable block

As the scripts are fully hand-crafted and in an easy to use excel format, it would be more effective to manage and maintain the test scripts.

Reports

Basic reporting. Being enhanced to provide interactive links.

Fully customizable & extendable.

Basic built-in reporting. Not much option to extend the contents & format of the report

O2T is slightly better off due to the enhancements and customization support for the reporting.

Scheduling

Requires enhancements to the framework.

Built in support

 

Data driven Testing

Basic support

Full Support

 
       

 

Other Aspects

 

Attribute

Open 2 Test

Selenium Web Driver

Telerik Test Studio

Comments

       

Load & Performance test support

Not supported

Built in support albeit with additional licensing cost.

 

Support for Non-browser based testing

Requires enhancement

Built in support

 
       

 

In Summary, the tools are pretty evenly matched on the attributes measured against. While Open 2 Test (Selenium) scores on the licensing cost and script maintainability, it requires some (one-off) enhancements & customization to the underlying framework to make this fully operational and integrated with the project delivery framework.

The recommendation is to apply weightage to the parameters that are used for comparison depending on the organization context (strategy, goals, immediate and long term delivery plans) and then arrive at a score card.

Advertisements

Open 2 Test

On a proof of concept I was working on for a new prospective client, I have had to get my hands on the Open 2 Test framework. This was an Open Source initiative sponsored by my (NTT DATA) organisation few years back and I never actually bought the value it could bring to an organisation. The main advantage as portrayed by the team is the interoperability across test tools (i.e. you could run the same test script using QTP or Selenium without any(?) changes hence prevents the organisation from any vendor lock-in. But seriously this alone doesn’t convince me to go learn a new scripting syntax, new tool etc. But when I looked at the tool hands on, I realised the true benefits of this tool.

This is a not a tutorial, but just discuss the key benefits of this framework. For detailed tutorial, there are many documentation available on the web site (http://www.open2test.org ).

Simplicity

The first thing that comes on top is the simplicity of the scripts. It uses a tabular form and works with s/sheet. The following screen shot shows a simple script.

This simple script launches a web browser (configurable), and performs a click and checks for values of couple of div tags. As can be see, it is very self-explanatory once the convention is understood, which is very minimal as well. This is a significant step when compared to the tool specific scripting languages. And the fact this can be done using s/sheet means, even business users could be trained to directly script tests rather than a need for a specific testing tools team.

Extensibility

Another key feature is the ability to extend the keywords and functionalities of testing as required by individual application and environment. For example, on the POC I was working on, there was a requirement to test for a specific text, which gets updated via an Ajax call. The use case was like adding a book to the shopping basket and it invokes an ajax call. And once the book has been added to the basket (at server), the ajax call updates the text on a particular (progress bar like) div tag to Item added to basket. The time lag between invoking the request and finally the text getting updated is unknown due to the nature of the ajax request. This need a wait function that need to wait till a particular text is updated on the div tag. This is supported by the underlying Selenium web driver. Hence I have introduced a new action keyword (waitfortext), which will wait until the element text changes to an expected text. There is an option to specific maximum timeout.

Looking at the source this was very straightforward as below.

It finds the element using one of identifiers or xpath. And then simply invokes the webdriver method to wait for its text value to be the one expected. (with max time limit)

Maintainability & Re-usability

Another key concept it uses is the decoupling of web elements and a logical identifier. In the test script you could refer to the web page elements using logical identifiers and then link these to the specific web elements using another s/sheet called Object repository. This gives a maintainability advantage and also allows re-usability. You maintain a s/sheet as below where you could associate logical names to the web elements.

Then object name would be used in the scripts, which prevents script updates whenever a page element changes. Also this enables re-using the object name across pages and serves as a complete data dictionary.

Open Source

Above all others, the framework is open source and very light weight. This allows complete freedom in terms enabling changes and customising as per specific needs. I don’t have to explain lot about the benefits of open source. However one down side is it doesn’t have a very active community, probably because of less awareness. But there is a small team still actively works on it and publishes periodic updates.

Summary

It is quite awesome, but it is due to its age, a bit old school. Now a days there are other open source alternatives available. .Net Canopy being one. But its ability to work with test scripts on s/sheet and be able to load & process data from s/sheet makes it very useful indeed. Perhaps this will require a bit of modernisation to be able to work with latest web technologies.

Pros

  • Conceptually strong
  • Open source
  • Others as above. (Simple, Extensible, etc)

Cons

  • Bit old style (although this could be improved)
  • The open source code isn’t very clean

 

ploeh blog

Ideas, Experiences & thoughts worth Sharing!!!

Passionate about data

Ideas, Experiences & thoughts worth Sharing!!!

You’ve Been Haacked

Ideas, Experiences & thoughts worth Sharing!!!

Swaroop C H - India, Startup, Technology, Life Skills

Conning people into thinking I'm intelligent. Since 1982.

Paul Graham: Essays

Ideas, Experiences & thoughts worth Sharing!!!

Martin Fowler

Ideas, Experiences & thoughts worth Sharing!!!

Scott Hanselman's Blog

Ideas, Experiences & thoughts worth Sharing!!!