While RAD helps software developers get early versions up and running very quickly, it causes testers many headaches. With each and every change is the opportunity to create new defects. The only way to find new defects is to perform a regression test which completely repeats a previous test and compares the results to find differences.
Two questions come to mind: 1) "Is it possible to completely test during rapid change?", and 2) "Which strategies can be used to test rapidly changing software?"
Is It Possible to Completely Test During Rapid Change?
Actually, no. However, that's a trick question because in most cases it is not possible to completely test software even in stable environments1. The essence of this question might be to ask, "Is it possible to test effectively during rapid change?" Can we expect to make the best use of people and other resources to test software? Can we expect to find the expected number of defects?
By observing projects using RAD, I have observed that a process for testing is essential to finding defects with any degree of effectiveness. Since the norm is to have no repeatable processes for most of what we do in building software, many people test in a RAD environment the same way they do in other environments - try a few cases here and there and look for problems.
Which Strategies Can Be Used?
It takes time to learn what works in each unique environment, but here are some general strategies that can be used for testing during rapid change:
1. First of all, accept the fact that you do not have the luxury of performing a six week test on software that changes every day. This means you will need to define a testing process that can be performed quickly and efficiently.
2. Perform a risk assessment. Knowing the level of risk is crucial, since you will need to prioritize your testing efforts to fit within a short window of time. The higher the risk, the higher the testing priority.
3. Automate your testing. Capture/playback tools help you perform repeatable tests in an unattended session. Good tools require a significant investment in software and training, but it beats working 24 hours a day. Some things to consider before automating:
4. You must have a working baseline version of the software to perform comparisons with future tests.
5. You must define business requirements, test cases and test scenarios. The tool can only record and playback based on what actions the user performs.
6. Test data is a key consideration. How will you maintain the test data? For example, if you use a capture/playback tool to add a record, a reply of the script will get a "duplicate record on file" error.
7. It takes time and a whole lot of spending money to integrate the tool into your organization. People need to be trained in how to use the tool. In addition, people need to be sold on the long-term benefits as compared to the short-term work required to setup the test scripts and test cases.
Testing during rapid change is not impossible, but it does require rapid response, working smart, and keeping track of changes. Organizations that have been unwilling to consider new technologies such as automated testing tools will not be able to effectively test during rapid change. It is like building a house with hand tools - sure it can be done, eventually.
Testing during rapid change also requires a new mindset of organization and processes. Tools alone are not the answer. There must be a process that can be executed quickly and makes the best use of people and time. It is arriving at the optimal combination of tools, processes, and people that is the challenge. To find out more about training and approaches for user acceptance testing, e-mail me.