Test Collab

8 free new testing tools which are making development teams more productive

Testing Tips No Comments

WOW, we’re seeing so much innovation in software testing this year.

A part of me can’t help but feel overwhelmed every time I see all these brand new awesome tools available which promises so much. More tools bring me more pressure, but they also bring insane improvements in how things get done in a team. That’s why – we never stop looking out for these new tools.

Every year new tools are doing exponentially more than their respective predecessor.

Imagine this:

About 8 years back, it took me over 100 hours to setup a simple CruiseControl build system from scratch for our very first product.

Today I can setup a build system from scratch with a new fancy tool with 100x more features in less than 30 minutes.

We keep coming back to finding out the right balance between this ‘exploration’ and ‘writing code’. So, today we bring you some of the most promising latest free testing tools which we’re really excited about.

#1 Sysdig

OK, Imagine you manage a SaaS app and you encounter a mysterious error on your production server which has slowed down response time of a request to 4 seconds. What do you do from this point on?

Most people would start with plain old ‘top’, then ‘strace’, ‘tcpdump’ and switch back and forth among these tools. You might even have 4 separate windows with all these tools running. Why not have a single tool that combines it all in a single view? Sysdig does exactly that!

It is strace + tcpdump + htop + iftop + lsof combined, so you can see everything with zero switching.

Not only this will help in production debugging, but also in development environment to locate key bottlenecks or to understand impact an application will have on OS.

We haven’t tried it personally yet, but it tops our to-try list.

#2 Browser sync

If you are a tester and you have ever done cross browser testing you know how boring it is to do same thing over and over again in multiple browsers, right?

And, with each web page, device and browser, testing time grows exponentially.

What if you could perform the test once and see multiple browsers in real-time? Browser Sync promises exactly that: Synchronized browser testing

With a little learning curve the payoff is huge.

#3 Concourse CI

Few years back when we installed Jenkins for first time, it opened whole new world for us. However as we learned more about deployment pipelines, we started hacking Jenkins more and more with all sorts of plugins. Jenkins core wasn’t just made to handle that.

But these brand new CI tools address the pipeline issue first-hand. One of them which we loved is Concourse CI.

The only downside is if you’re moving from a legacy CI system then you have to rethink your builds as pipeline and this takes time. But at the same time it is a good thing.

If the deployment pipeline concept is new to you, I highly recommend you and your team read this.

#4 EarlGrey

Many native iOS app developers face difficulties with existing test automation frameworks for iOS. The key problem in various iOS automation tools was synchronization.

So Google open sourced their own testing tool this year and the feedback has been great among testers.

EarlGrey is a native iOS UI automation test framework that enables you to write clear, concise tests. Tests written with EarlGrey offers much more stability and makes the process highly repeatable without extra efforts or hacking.

#5 Rep Sheet

If you run web apps in production you must have seen how growing number of bots can automatically attack these apps in much more sophisticated manner.

Rep Sheet might just be the cure for that. It is a plugin for Apache and Nginx web servers which record all user activity and aims to block offensive requests.

How it does that is by analyzing requests fingerprints, several predefined rules and user-defined rules. It also offers great reporting so your team is aware of what is going on.

#6 Ievms

If you have ever tested your web app in multiple versions of Internet Explorer, you know what kind of nightmare that is.

Well, now, you can be prepared upfront. This github project provides all the IE versions in VM so you never have to worry about setting up multiple Win environments for testing in IE again.

#7 Server spec

The new concept of Provisioning Testing is emerging after the automated provisioning servers took off as a result of DevOps movement.

‘Provisioning Testing’ means you are writing tests that make sure that configuration management tools like Puppet, CFEngine, Ansible have done their job well.

With Serverspec, you can write RSpec tests for checking your servers are configured correctly.

#8 Zipkin

It’s a tracing system but instead of the raw code it works on microservices. So as a result you get to see a beautiful step-by-step ‘waterfall’ graph which shows you how request passed on from service to service so you can troubleshoot latency issues.

Who should care?

Anyone involved with microservice architecture or anyone about to migrate from monoliths to microservices. (Speaking of migrating from Monoliths to Microservice – do not miss this thread on Hacker news)

 

Have any suggestions for this list? Please post in comments.

Manually testing Feature Branches the right way

Testing Tips No Comments

For many teams it is essential to work with different branches at same time so the main repository stays stable while development can still progress at a fast rate. Developers can create their own branches from the trunk/baseline and work independently on it. (Read more about that on Martin Fowler’s blog)

This creates a few problems when it comes to testing:

  1. With the changes that occur in branch, some test cases are affected too and needs to be changed.
  2. Multiple branches means maintaining multiple copies of a test case.
  3. Testing individual branches requires a tester to refer to the updated list of test cases only.
  4. Some branches will be merged to trunk/master sooner than later, that means at some point you will be merging these updated test cases to the trunk/master.

In this article we’ll describe how to solve these problems step-by-step.

Maintaining multiple copies and changes of test cases

To get started, you should have all your test cases in the trunk created in Test Collab as shown below. We’ll take a sample project as example here to make things clear.

The following image shows a typical suite dashboard page where existing suites and test cases are being displayed in a tree structure in the left section and details of the selected test case is shown in right section. Along with this information, there are few buttons getting displayed, which when clicked, starts the related operation/functionality like “Add new suite”, “Add new test case” and so on.

Suites and Test Cases Dashboard

Then create a new suite named ‘Branches’ where you’ll store different branches your team is working on. This new suite “Branches” should not have any suite as parent. Now create child suites of this suite one-by-one which represents different branches.

A branch will affect several test cases as it is actively developed. Say, our branch ‘My first branch’ affects all test cases in ‘Clients’ suite in different ways.

To manage this, you can simply copy whole test suite ‘Clients’ under ‘My first branch’. To do this, right click on the suite ‘Clients’ and a popup menu will appear. Select ‘Copy’ menu option from that popup menu. Depending on the numbers of test cases, it will take time to complete the requested operation and after that the suites/test cases list will be refreshed. Now the copied suite, in this case ‘Clients (2)’, will appear at the same level where the original suite ‘Clients’ reside and will have a copy of all the test cases which exists in the original suite.

Now, next thing which have to be done is to move the new copied suite under that suite which represents our branch i.e. ‘My First Branch’. To do this, we can simply drag-n-drop that suite ‘Clients (2)’ under the suite named ‘My First Branch’ or by editing that suite ‘Clients (2)’  and changing/selecting its parent suite as ‘My First Branch’ and save.

Now you have a while copied suite ‘Clients’ under ‘My first branch’. This can be done for individual test cases as well. This can be done either by clicking ‘Duplicate’ action link available on test case view or by ‘copy’ test case popup menu option which appears on right click on that test case.

Also you can create more than one branches and have multiple copies of test cases and suites related to the branches respectively.

After this is done, you can now proceed to make changes in your test cases independently in different branches. A change in test case or addition of a test case will live only under that branch and is completely independent of baseline or other branches. While changing a test case, also mention reason or comment in field ‘Note related to changes’ for later reference.

Testing Individual Branches

After you are done with managing the structure and copies of test cases in different branches, you’ll eventually need to execute these.
You’ll need to:
a) test trunk/master,
b) test individual branches

Testing trunk is simple: As there will be existing executions for suites and test cases  in trunk/master and those can be re-executed either by a tester or by automated testing tool (using test automation feature in Test Collab). Using test automation feature will also increase testing speed. To know more about test automation, visit the following URL.

http://support.testcollab.com/index.php?pg=kb.page&id=165

For branches, we need to create new executions and to easily recognize which execution belongs to which branch, also add branch name in execution title while creating an execution. For faster execution creation, execution templates can also be used. For example, template based on tags.

To test a different branch, you’ll need to select the suite under the branch name instead of first-level suite. In this case, ‘Clients (2)’ suite under ‘My First Branch’ suite should be selected.

After assigning test suites/cases to tester, the newly created executions are ready for test/run. The automated tests (if any) for the original test cases exists in trunk/master will not work properly for these modified/newly created test cases in branch. So, in this case, we need to run these executions manually by a tester. Rest of the test execution process will follow. For more information on Execution Management in Test Collab, click here.

Merging the test case changes after branch is merged

After complete testing, the branch will be merged into trunk/master. This will also require to update/reflect the test case changes, done in branch, into the trunk/master. Currently there is no way of automating the merge function and this must be done manually. We recommend opening the test case in multiple windows and compare them side-by-side.

Once you figure out the changes to make. You will need to update the test case (Edit link on test case view) at the baseline as indicated below:

After changes are done, you can see the updated test case with change log:

You can also run a side-by-side diff to see what was changed.

Step-by-step guide to integrate Ranorex Test Automation with Test Collab

Case Studies No Comments

Ranorex is easy-to-use test automation software (yet available for Windows OS only). A step-by-step wizard helps to set up the test environment and quickly get started.

For Windows application development and testing, it makes perfect sense.

Non-programmers can use the script-free drag & drop functionality, whereas professional programmers can use an API for C# and VB.NET to enhance their test suites and recordings.

It has a powerful GUI recognition covers all requirements in terms of accuracy and unique identification. It will recognize and find the element anyway even if the button’s shape or color changes. Facility to reuse code and action modules across multiple test cases with click & go functionality. This will save a lot of time when changing multiple test cases. Recording tests is very simple. Just press the record button, start your manual testing and It remembers all of the steps. Delete redundant steps with an easy-to-use editor.

So, why integrate Ranorex with Test Collab?

Three reasons primarily:

Reason #1: Connection between automated and manual testing

Many teams have a big disconnection between manual testing and automated testing. Both of these tests live in a different world. If you can keep them together, not only you improve your testing productivity but you also gain insights about your quality assurance process.

Reason #2: On-demand test execution for a selected a test case(s)

Almost every team has their automated tests tied up with their continuous integration servers, but there are times when you don’t need to run all these tests together.

Sometimes you just need to quickly test one key feature or functionality in your app.

Reason #3: Remote triggering automated test case and archiving results

What if you need to test important part of your application without setting up whole automation suite on your local PC? That’s when remote-triggering comes in.

By the end of this tutorial, anyone in your team will have easy access to run automated test case, without setting up anything on their PC.

OK, Let’s continue to installation…

Ranorex version & installation

For preparing this document Ranorex 5.4.5 version has been used.
For installation, please use the ‘setup.exe’ or self-extracting zip file ‘Ranorex-x.x.x.exe’ to start the installation, which will also install all required prerequisites.

Create solution/project in Ranorex

Create new solution/project

While creating a new Ranorex project it asks for the path where you want to generate the files along with the name of project and suite. A Ranorex suite may have one or more test cases.

Each Ranorex test case will represent a test case in Test Collab (this is how we achieve the ‘connection between manual and automated test cases).

The easiest way to create a test with Ranorex is to record a manually executed test scenario. The recorded actions like mouse clicks or keyboard actions are the base for building a robust test case.
There are different types of elements which can be used within a test suite like

  1. Folder – Used to group multiple test cases
  2. Test case – Represents a test case which can contain modules, a setup or teardown region or other test cases
  3. Setup region – Groups modules used to prepare a test case (e.g. start system under test, initialize a database, etc.)
  4. Module group – Used to group several modules into a reusable set
  5. Teardown region – Groups modules used to clean up a test case (e.g. deleting files generated during test execution, closing application, etc.)
  6. Code module – Automation module written in code
  7. Recording module – Automation module generated by recording

For more details, visit the following URL:
http://www.ranorex.com/support/user-guide-20/lesson-1-getting-started.html

Create test suite/case(s) in Ranorex

Before start recording, you need to ensure that your system under test is ready to start with the manual test execution. In addition, consider the following points in order to avoid too much work in cleaning up the recording and the repository afterwards.

  1. Do not run multiple instances of your application under test if that is not part of the test case itself.
  2. By default mouse movements are not recorded. For this reason please also perform a mouse click in situations like when navigating through menus.

Ranorex test suite/cases

Add [SETUP/TEARDOWN] sections for suite/test cases in Ranorex wherever required. For example, to close browser after test completion or close user session after login success.
Note: This document uses the Crowd Vox directory software which is a web appplication/CMS. We’ll test user registration and login web tests to demonstrate Ranorex integration with Test Collab for test automation.

Recording a Test and analyzing Recorded Steps

Ranorex recording module

Create a new test case and add a recording module to it. Open recording module and press/click “Record” button and start manual testing by providing website URL and selecting browser for testing.

Start Ranorex recording

Ranorex recorder popup window

A popup will be displayed during recording process which can be used to pause/resume recording and to add validation(s) wherever required. All the actions will get recorded into Ranorex recording module as actions. After completion of manual testing, click “Stop” button on that popup.

Recorded actions

After this review/examine the recorded actions as well as repository elements which are basically DOM elements clicked/used during manual testing. Remove unwanted actions and change actions as per requirements (if any, for example – delay, mouse clicks, key sequences, etc.).

Use variables in place of values which enables to repeat the test with different values. In addition each action item is connected to a repository item which represents a UI element (text boxes, radio buttons, buttons, etc.) used during recording. For more information on recording tests, please visit the following URL:

http://www.ranorex.com/support/user-guide-20/lesson-5-ranorex-recorder/before-starting-to-record.html

Executing the Test using Ranorex

In order to execute/test the recorded test case you need to switch back to the Ranorex test suite file. Just click on ‘Run’ button to execute the test suite with your recorded test. During the execution Ranorex simulates all user actions (mouse movements and keyboard events including delays) in order to test the application in the same way a tester would successfully do it.
Using variables in place of constants in Ranorex tests is a key to reuse/re-execute same test  with different values by passing them as command parameters.

Run Ranorex recorded test

If you want to execute any particular test case at a time then select that test case and right click on it and a popup menu will be displayed. Click on “Run selected test case” menu option.

Run individual test case in Ranorex

Before starting a test execution, Ranorex first compiles/build that project and inform about errors (if any). In this compilation process on the basis of the project configuration, an executable file (.exe) and other related files where generated in the bin/Debug or bin/Release directory under Ranorex project path.

After executing the test with Ranorex Studio, it automatically opens the generated test report file (*.rxlog) which shows whether the test run was successfully or not. A test is reported as failed when any validation fails or the linked UI element does not found as per action recorded during manual testing.

Note: For more information on executing Ranorex tests using command line and configuring the test automation environment using command line arguments, visit the following URL.
http://www.ranorex.com/support/user-guide-20/lesson-4-ranorex-test-suite.html

Ranorex test report - pass

After test completion, a report will be displayed (if report is enabled in project configuration).

Ranorex t2est report - fail

Deploy Ranorex test for test automation in Test Collab

The required files for executing Ranorex tests on runtime machines are located in the “bin” folder of the Ranorex installation path. These files can be found easily on machines having a valid Ranorex Installation using the environment variable %RANOREXPATH%. Please copy the whole contents of the “bin” folder (including all subdirectories) to a target folder (<TargetFolder>) on the runtime machine.

Note: It is crucial to deploy the Ranorex Assemblies with exactly the same version that was used to compile the test executable. For more information on how to deploy Ranorex tests on another machine which do not have Ranorex installed, please visit this link.

Ranorex output folder

Ranorex generates the files needed to run automated tests in the “Output Folder” (by default <ProjectPath>\bin\Debug) of a test suite project. You can directly access this folder by right-clicking the current project and choosing “Open Output Folder” in the context menu.

It’s recommended to clear your output folder before and then trigger a new build (Build -> “Build Solution” or hot key F8).

Ranorex build options

  • Required files for executing Ranorex test automation from output folder:
    1. Executable file (*.exe)
    2. Test Suite File (*.rxtst)
  • Additionally required on the base of your project needs:
    1. Ranorex Module Group (*.rxtmg), only necessary if modules groups are in use
    2. Test Data (*.xlsx, *.xlsb, *.xls, *.csv, *), only necessary for data driven tests and if data source files were added to project
    3. Module Libraries (*.dll), only necessary if module libraries are linked and embedded functionality is referenced in the Test Suite
    4. Sub folder “RepositoryImages” including all files, if exists

Finally copy all the files needed to the target machine and place them in the folder where all Ranorex Libraries and Assemblies are already located (<TargetFolder>). Of course, by copying the whole output folder (“bin\debug” by default), you will ensure you have everything possibly required for running your test.
In short, it is best to copy the complete output folder (e.g. ‘bin/debug’) to the target machine.

Enable test automation and create remote executor in Test Collab

Log in to your Test Collab instance with an administrator user account and open “General Settings” page. On this page mark the checkbox as selected saying “Enable Test Automation” and save.

Enable test automation setting

After that, refresh the page and a new tab/option “Remote Executors” will be displayed under “Settings” menu. Add remote executors as per your requirement.

Remote executor in Test Collab is simply a machine where your automated test runs.

Next step: Adding a remote executor

To add a machine with remote executor to Test Collab: You can do it From Settings ->Remote Executors. Adding remote executor will need following information:

  • Machine name: This is the name which will represent machine in the application.
  • Machine API Key: This can be any random series of alphabets that will be used for authentication of remote executor. This field should be unique for each remote executor.

Add remote executor

For more information about installing/configuring Test Collab for remote machine visit the following URL:

http://support.testcollab.com/index.php?pg=kb.page&id=165

Project wise test automation settings

Login to your Test Collab instance with an administrator user account and select the project for which test automation settings has to be done. Click on “Settings” menu and then select “Automation Settings” option under it. Enter values for the following fields and save automation settings.

Project wise test automation settings

  • Base Dir – Full path to the directory on remote machine where automated test cases has to be executed by remote executor program. In this case, it will be a target folder in which Ranorex test executable file along with other required files generated using build project and Ranorex installation folder bin files exists.
  • Command template to execute individual test case – Command to be executed to perform automated test execution using Ranorex. In the following example
    C:\tcm_test\crowdvox\cv.exe /tc:[[ranorex_test_case_name]] /rf:report.rxlog
    Let’s break it down:
    cv.exe is a Ranorex executable file (generated using build project option in Ranorex) which will trigger our automation
    /tc
    (to specify which test case to execute – [[ranorex_test_case_name]] will be replaced with the value saved in Test Collab test cases for automation) and
    /rf (to specify report file name other then default report file name which will be generated during automated test run). Arguments should be enclosed within 2 capital brackets for example [[ranorex_test_case_name]].
  • Command Parameters – Arguments (if any) which are entered in Command template, has to be entered here by separating each of them with a new line. For these parameters, values will be entered for each test case and will be used in test automation process.
  • File(s) to collect after test execution – These are the files which will be looked for, and archived after automation runs.
    New line separated list of files with full path (if any) which has to be collected from remote machine. In this case, there will be 2 files which are Ranorex report files (report.rxlog and report.rxlog.data) which will provide information about test result executed automatically by remote executor.

Automate test cases

After saving project wise test automation settings, we need to provide the values for command parameters (if any) for each test case in Test Collab. Open the Test cases list page and select the test case(s) which needs to automate. A new action link “Automate” will be displayed on test case view along with other actions (like Duplicate/Edit/Delete – based on user rights).

Automate link on test case view

On test case automation params page, provide the desired values for command parameters as per automation testing requirements and save the data.

For example, “Login” and “Register” are entered as test case automation params which will be used to execute a particular ranorex test case during test automation for that execution case. In this way, we can add as many test cases in a single Ranorex Suite/Project and use them as per requirement.

Test case wise automation command parameter

Test case wise automation command parameter

OK, Now it’s time to trigger these tests remotely…

Create test execution in Test Collab
Create a new test execution in Test Collab as per requirements and select test suites/cases for that execution and assign them to Remote Executor to automate the test execution. For more details on test execution/plan, please visit the following URL.

http://support.testcollab.com/index.php?pg=kb.chapter&id=17

Create test execution in Test Collab

Assign test cases to remote executor

After assigning test cases to remote executor, click on “Save” button to complete the execution assign process.

Executions index

Once a test execution has been assigned to a remote executor, the remote executor fetches the execution case information from Test Collab along with automation parameters values/settings for that test execution case and starts the execution of Ranorex executable and providing the parameters to it (if any).

Automated test run

Image-1 showing automated execution of test cases using Ranorex on remote machine by Test Collab remote executor.

Automated test run
Image-2: Automation running in progress

After completion of Ranorex test execution, the remote executor program collects the result and passes it back to Test Collab server along with the requested data (like report file or any other file generated during test execution) and on Test Collab server that particular execution case status will be set as pass/fail according to the information provided by remote executor.

Executions index

Detailed Execution view after completion of automated test run

Execution view after automated test run

Execution case individual view

Execution case view after automated test runAutomation result

On this page, in “Logs” section, the output during automated test execution using Ranorex suite/project executable is being displayed (green text) along with the test result (Pass).

Execution case view after automated test runAutomation result with attchments and other details

At the end of the “Logs” section, other information related with the execution will be displayed like execution start/end time and total time taken in that particular execution run along with the attached files generated during automated execution process (report.rxlog and report.rxlog.data files).

Key takeaways:

1. Connect your manual and automated testing together at one place for better control.

2. Automated test cases isn’t just for developers to trigger or advanced testers, give power to your whole team by enabling remote control.

3. Have granular control over automation. Enable your team to run one single test case if needed without having to run whole build.

I hope you like this tutorial. Like we integrated Ranorex with Test Collab here, any test automation tool can be integrated as long as it supports command line. What would you like to learn next? Scaling test automation, Integration with Selenium, Google’s latest EarlGrey, context-driven testing with Test Collab… Let me know.

9 Mobile App Testing ideas that will speed up your releases

Testing Tips No Comments

Testing mobile apps is a big challenge because of complexity involved: you have multiple devices carriers, battery life, limited space and so much to take care of. Go through this simple list to speed up your mobile app testing and shorten the release cycle.

#1 Top 7 challenges in mobile app testing


http://www.thinksys.com/blog/top-7-challenges-in-mobile-app-testing/

#2 Thinking before you start testing


Before you jump straight to testing, it is important to get some perspective. This article does that and does it well. You can learn what to expect, what lies ahead, what you’ll have to manage, what you’ll be up against. You’ll know how critical testing will be for your success, so it doesn’t become a liability for you and you’ll have the motivation to push it further.
http://www.infoq.com/articles/mobile-app-testing-the-secret-to-the-perfect-app

#3 Know different type of mobile tests


Because of the complexity involved in mobile app testing, you should know what kind of tests you’ll be working with. That will help you focus on areas which are relevant for your app at the right time. When you are trying to launch first version, you will have different priorities than when you are trying to scale an already successful app.
http://www.business2community.com/mobile-apps/know-5-types-mobile-app-tests-01390827

#4 Rely on Real devices, not just emulators


Emulators are great while development but do not rely only on them for testing your app. You would think running app on emulator would be same as that of device, right? Wrong! On real device there are many things that can go bad including but not limited to graphics, internet connection, access to file system and so on.
http://testdroid.com/news/rely-only-on-real-emulators-vs-devices

#5 Testing as part of your mobile app development


Not just for mobiles but for any application or environment that is constantly changing. Detecting and fixing a bug early in development would save you 10x or 100x time over the bug discovered late in production environment – not to mention the loss in business that a simple bug can result. That is why you should always be testing. Testing should not be seen as a phase but as an ongoing process.
http://testdroid.com/tech/test-early-test-often-testing-as-part-of-your-app-development

#6 Five ways to speed up mobile app testing


You don’t want to be stuck in endless testing loops when your competitors are launching new versions every week. Mobile app users are more demanding and they expect releases more frequently. Equip yourself with the knowledge and ideas on how you can speed up your testing cycle.
http://devops.com/2015/06/02/5-ways-to-speed-up-mobile-app-testing/

#7 Dos and donts of mobile app testing


Simple list of do’s and don’ts of mobile app testing. Some of these are no-brainers but you should keep this list close to everyone in your team – from testers to developers, so they are in agreement what your priorities are.
http://mobilbusiness.se/blog/thedos-and-donts-of-mobile-app-testing/

#8 Utilize test automation


If you are constantly releasing new versions of your app, you know how difficult manual testing can get. Not only it is more expensive but it is less reliable too. You should utilize test automation for cutting your testing time so you can focus on other important things.
http://testdroid.com/tech/increase-efficiency-and-productivity-with-test-automation

#9 Twelve mobile app testing frameworks


Evaluate these testing tools/frameworks which will surely do wonders to your mobile app. Learning each tool is a an investment so choose wisely – you do not want to use each and every tool but you should try them at least once to decide what works best for you.
https://codegeekz.com/12-best-mobile-app-testing-frameworks/

New feature added: Re-execute test executions

Announcements No Comments

You can now re-execute a test execution instead of creating it all over again. Here’s how it works:

Step 1: Hover on test execution you wish to re-execute and click re-execute.

Step 2: Select statuses which you want to re-execute

Step 3: Execute cases

This feature can save you a great amount of time in creating duplicate executions.

Test Collab has been featured in Pivotal Tracker App Bazar

Announcements No Comments

After announcing our integration with Pivotal Tracker issue manager and new features, we’ve been published as featured app in this edition of Pivotal Tracker’s App Bazar.

Thanks to the guys at Pivotal Tracker and all our customers for the support.

Test Collab v1.12 released: Introducing Test Plans, Improved Requirements Management and coverage reports

Announcements No Comments

We’re proud to announce our latest release Test Collab v1.12, which introduces Test Plans, an in-built requirements management engine, new coverage reports: for entire project, version-wise and module-wise and several enhancements and bug fixes.

Test Plans
You can now contain multiple test executions under one entity, i.e., test plan. Besides acting as a parent for multiple test executions, you can also use test plans to mix-and-combine multiple configuration values for creating multiple test executions. Consider this example:
You have one web app that needs to be tested with browsers: IE, Chrome and Firefox on OS: Ubuntu and Windows.
With Test Plans you’ll just select the configurations and cases to execute, it’ll automatically create multiple 3 x 2 = 6 test executions.
Learn more about Test Plans.

In-built Requirements
So far you only had an option to use external sources or your issue manager as requirements engine, but this limited many of you to do things. So we decide to make our own requirements manager. You can create requirements in form of features and scenarios, contain them in versions or group them in modules. You can also track coverage report for requirements project-wise, version-wise of component-wise.
Learn more:
Requirements Management
Coverage Report
Managing Versions and Modules

Both these features open up a lot of possibilities for teams. Stay tuned on our blog for upcoming detailed examples and how-tos.

Test Case Versioning with Test Collab

Testing Tips No Comments

During your application life cycle a lot of new changes are made to existing features, and when these features change, so does your test cases. Do you know Test Collab stores all revisions of your test case? So every time you make a change in your test case, the old revision is automatically stored. This means you can keep track of such changes over time.

At times this information can be of great help, say, you are a test engineer and you are looking at the executions from 3 months back for ’Feature A’, and you want to know how exactly this test case was executed back then. But there have been a lot of improvements in ’Feature A’ in your application in last 3 months, and so obviously test cases associated with that feature are also changed several times.

Test Collab will take care of such scenarios by automatic versioning of all your test cases and storing each revision. It will also store reference of the test case’s revision # which was executed, so you can view the same.

Our latest addition: Execution templates, because we love DRY

Announcements No Comments

Creating same kind of test executions is a hassle, especially when you find yourself selecting same set of cases again and again. Test Collab introduces a feature of Execution Templates with this release, and as the name suggests, based on these templates new Test Executions can be created quickly.

Test execution creation process requires the user to select the set of suites having the test cases that he wants to be executed and the tester who will be responsible to execute them. This is quite simple and straightforward process, but what if the user has to create the test executions with same set of test suites to be assigned to the same group of testers, at the time of every product release (say for regressions tests or integration tests) ? Execution templates come handy at the times where the similar kind of assignment process is to be repeated time and again.

With execution templates you can now have one or more test case selection criteria defined on the basis of test suites and / or tags. Each criterion can have one or more testers selected for it.

Once an execution template is defined, new test executions can be created on its basis and they will automatically have test cases matching the selection criteria defined. All desired test cases will also get automatically assigned to designated testers.

More details on the feature are available here.

Pivotal Tracker integration now available

Announcements No Comments

Today we’ve released Pivotal Tracker integration with Test Collab. Pivotal tracker is lightweight, agile project management tool for software teams. See Integration guide for more details.

Page 1 of 41234