Screencast: Test Automation with Test Collab

Ok first things first: What’s the main problem with most test automation techniques are: they do not offer very well interface, so you don’t have a good workflow around them in order to execute and extract meaningful information from them, most of time these tests end up being forgotten in build process or gets too old with time without ever being updated. Nor are they integrated well around your manual testing process. Writing automated test case is too damn easy, but difficult part is collecting insights and meaningful data from them over time. See this screencast and find out how Test Collab can solve your test automation worries: (small note: this is my first screencast, so please excuse bad voice quality)

To enable this feature in your account, go to Settings > General Settings, then check ‘Enable Test Automation’ and hit ‘Save’. Refresh once to see the ‘Remote Executors’ link under ‘Settings’ tab.

About the Author

Abhimanyu is founder of Test Collab, a test case management tool. Test Collab makes your testing more productive and efficient by enabling teams to collaborate in real-time.

  • Will

    That looks nice – however I’m not sure how to get started. Do you have a tutorial?

  • Tarun

    I use selenium like you did in this video but all my test cases are HTML, can I still use automation feature?

    Another question: Is there anyway to randomize the test execution across all online machines?

  • Abhimanyu Grover

    @Will If you have an account with us already – simply go to the knowledgebase from help page, then see “How to install remote executor” article.

    @Tarun Yes, you can. Selenium HTML tests can be triggered by following command:

    java -jar selenium-server.jar -htmlSuite “*firefox” “localhost” “C:\project\test.html” “C:\tmp\results.html”

    Specify this command in your project’s test automation settings. And specify “C:\tmp\results.html” as file to collect post-execution as an artifact.

  • Adam

    Isn’t it possible to schedule the test runs automatically? I tried finding it in my account – but couldn’t find anything related to schedules.

  • Abhimanyu Grover

    The scheduling feature is not developed yet – but its on our list.

  • Guest1

    My company uses SauceLabs to test our web application. I am curious as to whether integration of Selenium/SauceLabs VM/TestCollab is possible?

    • Abhimanyu Grover

      Yes, it will be possible – as you can configure Test Collab to run any command from your test slave machine. You need to know what command you issue to run the test and then specify it in Test collab automation settings.

      Just to be clear, this would require a machine on your end which will act as bridge between Test Collab and Saucelabs, as Test Collab directly cannot communicate with it.

  • Wayne Sinclair

    Can Test Collab interface with Cucumber? I am thinking of using Test Collab to store test results from Cucumber.

    • AbhimanyuG

      Cucumber integration is not supported out of the box, but you can do this with our API. Some programming will be required based on your requirements.

      There are two methods to approach this, a) Using cucumber hooks to post results to Test Collab API as tests are executing, b) Run cucumber test with option “–format junit”, Once all the cucumber tests are executed, parse the junit xml file and post test result data to Test Collab API.

      Since doing (b) is much easier, I’ll explain it further here:
      – You need to make sure that your cucumber scenarios are mapped with Test Collab cases. So you need to alter scenario titles like this:
      Scenario: User is redirected to the requested page #testcase: 407
      When …
      Then …
      “#testcase: 407” this part indicated the test case ID stored in Test Collab.
      – Create new execution at Test Collab:
      – Run cucumber with “–format junit”, this will create a junit formatted result file.
      – Parse the junit file (test case ids can be extracted from scenario titles) and post results via Test Collab API

  • Hans

    The company that I work for is about to introduce Test Management tool and Test Collab is one of the tools that we can try to consider. We are also in the midst of choosing QTP or TestComplete as our automation tool. Can Test Collab integrate with these tools?

    • Abhimanyu Grover

      Hi Hans,

      Yes, Test Collab can be integrated with both of these tools.

      • Hans

        Can you please provide me on how to do that? Thanks…

        • Abhimanyu Grover

          Sure, do you have any preference between QTP or TestComplete?

          • Hans

            I would start with simple script using AutoIt (I’ll try this first because this tool is light). I have followed it based on the video here but my script won’t run. And sometimes, my Test Execution indicates that the it performed the script – but the script itself did not actually run… (By the way, I’m testing this with both TestCollab and Remote Executor are in the same machine) — Thanks

          • Hans

            Just ignore my previous reply. I don’t know what I just did but it just run the script (only it takes some seconds before it ran).

            However, my question is how can TestCollab determine the status of its test? Because it appeared all the test cases were all passed. I designed one script that is expected to fail.

          • Abhimanyu Grover

            Hans, it checks the exit code of the process. Exit status = 0 means pass, any other status means failed. For AutoIT, you can refer here:

          • Hans

            Oh great! I finally make it worked… Just the exit code is missing on my script.

            We haven’t decided yet on which Test Management tool we’re gonna use but so far I got something from TestCollab. We have not decided which test automation tool as well but let’s say QTP or TestComplete, how to do that?

          • Abhimanyu Grover

            Great – it’ll be much easier now on as you’ve already done so much so far. Regardless of any automation tool you choose, all you’ll need to do is locate command line method to execute tests and then configure it at Test Collab. And by default most of test automation tools comes with command line support.

          • Hans

            Thanks again… I have now figured that out.

            By the way, I sometimes encounter the script won’t run after defining test execution from TestCollab. The test status still untested. I checked on the Remote Executor service, it was up and running.

          • Hans

            Another issue:

            I created a script using TestComplete. My script has been executed after creating Test Execution. However, it keeps on looping running the script. It would only stop once I deleted the entry on from the Test Execution list.

          • Abhimanyu Grover

            Can you contact us with details like your URL etc.?

          • Hans

            Before I’ll do that, just a quick question…

            After a computer restart and then try to run a test execution, my automated script won’t run automatically (I’ve waited for like 5 mins, nothing happens). What I did is to run the batch file from bin folder (remote_executor.bat). Since then, it would be working fine.

            Do I need to do that every after computer restart?

            By the way, all the issues mentioned above (test execution looping and script not running) are now working fine. The former, I have to fill up something on “File(s) to collect after test execution” field. It was empty on my Automation Settings. And the latter, as mentioned here in my comment.

          • Hans

            Maybe I’ll have to put this as request for easy tracking… Thanks

          • Abhimanyu Grover

            Thanks, that’ll be great.

          • Hans

            Seems to be a problem as what the support said… But he assures the fix will be carried out on the next release of Remote Executor… Thanks again for a very quick response.

            Well recommended tool!!!