Architecture Posts

Ideas, Experiences & thoughts worth Sharing!!!

Open 2 Test Vs Telerik Studio

Recently I had to do a test tool evaluation in trying to establish the right approach for a client. This involved comparing the Open 2 Test framework using Selenium web Driver and the Telerik test suite which the client already had in place for some of their projects. The objective is to aid establishing the key drivers and customizing a solution that achieves the maximum benefit for the investment. This is specifically done for testing web applications, hence the focus is on web functional testing, and however I have also listed other aspects separately below.

Web functional testing

 

Attribute

Open 2 Test

Selenium Web Driver

Telerik Test Studio

Comments

Cost

Free. Open source software based. Uses common office software like MS Excel for scripting tests.

Commercial product licensed per duration. (To check if it supports unlimited users?)

Well open source is not all free, and will require an initial investment to extend and customize the tool as required. So this need to be considered in the right context.

Skills & Learning Requirement

Predefined set of simple keywords used for scripting.

UI based scripting language.

Record & Play mechanism.

Both are intuitive to large extent and may require some amount of learning & basic training.

Integration with TFS

Extensible. Java based, hence could leverage on numerous open source libraries available.

Built-in support

While Telerik supports full integration with Visual Studio & TFS, O2T could be extended & customized to integrate as per the specific requirement. (Logging defects in a specific TFS project, etc)

Multi browser support

Existing support for IE & FF. Could be extended for others, but will require framework enhancement effort.

IDE support only for FF

Built-in support for all major browsers for both recording (scripting) and testing.

Telerik’s feature enables cross browser testing automatically.

Test pack maintainability

Fully hand-crafted scripting, hence relatively easier to update & maintain.

Decouples the page DOM elements using object repository, hence easier to maintain page element changes.

Supports script re-usability.

Recorded scripts hence will require effort to update for maintenance.

Allows in-line script editing.

Allows grouping of tests into a re-usable block

As the scripts are fully hand-crafted and in an easy to use excel format, it would be more effective to manage and maintain the test scripts.

Reports

Basic reporting. Being enhanced to provide interactive links.

Fully customizable & extendable.

Basic built-in reporting. Not much option to extend the contents & format of the report

O2T is slightly better off due to the enhancements and customization support for the reporting.

Scheduling

Requires enhancements to the framework.

Built in support

 

Data driven Testing

Basic support

Full Support

 
       

 

Other Aspects

 

Attribute

Open 2 Test

Selenium Web Driver

Telerik Test Studio

Comments

       

Load & Performance test support

Not supported

Built in support albeit with additional licensing cost.

 

Support for Non-browser based testing

Requires enhancement

Built in support

 
       

 

In Summary, the tools are pretty evenly matched on the attributes measured against. While Open 2 Test (Selenium) scores on the licensing cost and script maintainability, it requires some (one-off) enhancements & customization to the underlying framework to make this fully operational and integrated with the project delivery framework.

The recommendation is to apply weightage to the parameters that are used for comparison depending on the organization context (strategy, goals, immediate and long term delivery plans) and then arrive at a score card.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

ploeh blog

Ideas, Experiences & thoughts worth Sharing!!!

Passionate about data

Ideas, Experiences & thoughts worth Sharing!!!

You’ve Been Haacked

Ideas, Experiences & thoughts worth Sharing!!!

Swaroop C H - India, Startup, Technology, Life Skills

Conning people into thinking I'm intelligent. Since 1982.

Paul Graham: Essays

Ideas, Experiences & thoughts worth Sharing!!!

Martin Fowler

Ideas, Experiences & thoughts worth Sharing!!!

Scott Hanselman's Blog

Ideas, Experiences & thoughts worth Sharing!!!

%d bloggers like this: