Test Specs for manual and automated test
Someone ask me to write as a Proof Of Concept (POC) an automated test to validate the login process.
The manual specification was written in this way, using TestLink
Just one test case with six steps. When you will test manually this seems a good approach instead of writing six small test cases.
In order to organize things in a better way, you can refactor this creating two test cases
- AC-23 Validate Forbidden Login Process (RED login)
- AC-24 Validate Valid Login Process (GREEN Login)
How to approach the automated test
My tool of choice is test cafè, with the addition of TestCafè Studio to be used as an assistant in the test script development.
My first approach to this kind of tests, where there is a lot of similar processing/duplication is to try to make it a little bit data driven.
I can create a JSON file with an array of objects, where each object is one of the steps described in the specification, i.e.
You can loop over the array, executing a simple common code like this
On issue with this code is that when the validateLoginFailure() returns false because we are not anymore in the Login Screen if we do not ‘fix’ this situation the next step (next iteration) will fail. This has happened during the debug of the test script implementation, because I was able to login with a disabled login.
Because I wanted to use the ‘loop’ approach I can not use the expect() test cafe method, because expect() stop execution on expect() failure. I wanted to execute all steps, getting execStatus true/false for each step, and then creating an overall status true/false, providing the tester the details of failed steps.
Due to my approach I need to understand in what screen/state in addition to the right one (i.e. login failed -> system will remain in the login screen) I will land if the login block login fails.
Theoretically I can expect:
- the main application screen
- a crash
If these are the only alternatives, this means that the only thing I need to do is to have a ‘recovery code’ inside the if (step.execStatus == false) {
Test Cafè solution — Data Driven Tests
Here the link to the Test Cafè documentation.
More links: Test Automation University, Series on Dev Tester Site.
Applying this approach, my code has been changed to this:
