Test Specs for manual and automated test

Someone ask me to write as a Proof Of Concept (POC) an automated test to validate the login process.

The manual specification was written in this way, using TestLink

Test Specification using TestLink

Just one test case with six steps. When you will test manually this seems a good approach instead of writing six small test cases.

In order to organize things in a better way, you can refactor this creating two test cases

  • AC-23 Validate Forbidden Login Process (RED login)
  • AC-24 Validate Valid Login Process (GREEN Login)

How to approach the automated test

My tool of choice is test cafè, with the addition of TestCafè Studio to be used as an assistant in the test script development.

My first approach to this kind of tests, where there is a lot of similar processing/duplication is to try to make it a little bit data driven.

I can create a JSON file with an array of objects, where each object is one of the steps described in the specification, i.e.

You can loop over the array, executing a simple common code like this

On issue with this code is that when the validateLoginFailure() returns false because we are not anymore in the Login Screen if we do not ‘fix’ this situation the next step (next iteration) will fail. This has happened during the debug of the test script implementation, because I was able to login with a disabled login.

Because I wanted to use the ‘loop’ approach I can not use the expect() test cafe method, because expect() stop execution on expect() failure. I wanted to execute all steps, getting execStatus true/false for each step, and then creating an overall status true/false, providing the tester the details of failed steps.

Due to my approach I need to understand in what screen/state in addition to the right one (i.e. login failed -> system will remain in the login screen) I will land if the login block login fails.

Theoretically I can expect:

  • the main application screen
  • a crash

If these are the only alternatives, this means that the only thing I need to do is to have a ‘recovery code’ inside the if (step.execStatus == false) {

Test Cafè solution — Data Driven Tests

Here the link to the Test Cafè documentation.

More links: Test Automation University, Series on Dev Tester Site.

Applying this approach, my code has been changed to this:

Electronics Engineer from University of Buenos Aires. TestLink Team Leader. Methodology Manager @TESISQUARE