Developing a Regression Testing Strategy

Before agile, regression testing was simple. The testers would take all the ideas for regression testing that were created during a release cycle, add them to the existing ideas, and then run each one until the entire stack was complete. The time it would take to ship software was easy to calculate when it was released once a quarter or every six months. Software teams today aim to release software every two weeks or faster.

Agile teams test every function during the release cycle. They may do some light poking one day before shipping and then give their approval. They could also do nothing, and the software still ships. A regression testing strategy is a better approach to this problem.

How to Develop a Regression Testing Strategy

Each new software update can introduce new problems. A user might be presented with a new page, and asked to submit a date format that the programmer didn't expect. The user might get an error message on the page, explaining what they don't understand or care about. These are all risks that are easily imagined and where feature testing during sprints shines. 

These scenarios can be performed by testers, who will then report the issues. Programmers can review the code and fix any problems. Then, they can create automated checks that run along with the build to help the team quickly find out if there are new ones.

At the end of every sprint there are a lot of small changes and new features. Each of these was tested a bit on its own. It seems that everything is fine now. It could be, but it is possible. The interactions between each of these changes with the rest of the product can pose a risk.

Software is not an isolated entity. The authentication library interacts directly with user profiles. These interact with screen design, data access, and reporting. One feature's tendrils can touch another. Sometimes, unexpected ways are discovered. These surprises can sometimes be discovered through regression testing.

How to create a regression testing strategy

Socializing regression testing information via a 'regression board' is one way to address the problem of short time for regression testing.

Each member of the testing group can post-it notes a change that they have worked on during sprints and stick it on a wall. The notes are arranged in a top-down order based on which changes should be tested first. This list can be ordered according to perceived risk or importance to the customer. The CEO could also order it based on that information. 

Based on their knowledge, other members of the team can slightly rearrange the cards. Because a programmer knows that changing a user profile can potentially impact a demographics report, she might add a card to the deck. Based on their unique project perspectives, different team members contribute their knowledge to the regression test board.

The board can be referred to by the test team when the feature work has been completed and it is time for pre-release testing. There are many decisions to be made if there is more than 12 hours of work on the board, provided everything is in order and testers don’t spend too much time investigating and reporting bugs. Managers can decide to remove a feature from the list. Other non-testing staff can also be hired to assist with low-risk features or extend the release. Managers can be more educated and make better decisions.

A well-organized and clear list of regression testing tasks create a shift in the conversation from "There's not enough time" to "This team knows what is important and how many hours are available."

The technical side of a regression testing strategy

Technical is the other side to a successful regression testing strategy. Programmers can create automated checks while developing new features. Culture shifts eventually can lead to a situation where a feature cannot be considered ready for commit until the programmer has completed some check building. Although this is not testing, it can prove that code changes work as expected. This is intended to reduce the risk of changing code and provide a system for change detection.

This is one example of a strategy to test regression before releasing new code to production. Next, assess the way that the development team prepares to release a new version and identify areas for improvement. Many teams already have a strategy but don't know how to use it.