Risk assessment and risk management are two topics you hear a lot about. However, we seldom assess people’s tolerance to risk as it pertains to software projects. Without understanding risk tolerance, risk levels don’t help us make reasoned software project decisions. This same thought can be applied to other areas of risk besides software projects.
 
A great example of this is my financial advisor. When I first started working with him, he had both my wife and me complete a simple five-question survey to determine our tolerance for investment risk.

The questions were simple, such as “How anxious do you feel when the stock market goes down?” We were asked to rank these types of questions on a scale of 1 to 5.

We arrived at a total score, which helped our financial advisor to know which types of investments are the best ones for us, given our comfort level, investment goals and timelines for things such as sending the kids to college or for retirement. It also helped protect both him and us by knowing when an investment vehicle was too risky for us. Should we consider investing in a fund with too much risk, we would know that we were exceeding the tolerance levels we had indicated before we started looking at the potential high rates of return of the more risky fund.

Now, let’s apply this to software projects.

Every software project has investors. They are called “stakeholders” because they have something at stake – funding, credibility, the capability to do their job, and even the job itself.

Each stakeholder has a comfort level for how much risk they are willing to tolerate, but they are hardly ever asked specifically about their comfort (that is, tolerance) levels. In fact, my observation is that many companies take extreme risks with stakeholder investment and never think about getting their buy-in in terms of risk sharing or risk acceptance.

This means that if the project gets into trouble, the stakeholders don’t fully understand the impact to them until it is too late. It’s like investing in a hot stock and then seeing it crash, then kicking yourself because you didn’t objectively evaluate the risks and take a more sensible position.

This also means that the project manager is at risk of getting all the blame because the stakeholders may have transferred all their trust (and risk) to the project manager. Hospitals don’t even do this!  If you or a loved one has ever had surgery, you are asked to sign a waiver stating that you fully are aware of the risks of the surgery.

My friend and colleague Burt Greenberg brings out some good points that in a surgery, you are going to get counsel from the doctor about the both the benefits and risks of the surgery. This information is based on past performance and experiences. So, to accurately counsel someone in terms of risk, we need good history and ways to analyze that experience to determine the level of risk. This is where risk assessment comes into play.

As Burt notes, “To have risk assessment for IT projects, wouldn’t we need to share some history as well?  Wouldn’t we need a means to categorize projects early as to ‘success rates’ of comparable projects in the past? All this gets to our ability to provide the stakeholder with some objective insight about the project.  (For example, likely to stay within x% of budget, delivery date, functionality, etc.)  My sense is that we want the stakeholder to understand the risk, but we cannot provide the information by which their tolerance for risk can be determined.”

The catch is, risk by its very nature is uncertain. Whenever we assess risk we also need to understand that we may be off by a little or a lot. I have written an article (The Risks of Risk-Based Testing) that goes into some reasons we assess risk inaccurately.

Burt’s thoughts reflect the balance between understanding the risk and also understanding the tolerance for that risk. The difference in these two is normally seen when someone starts to complain about a loss. At that point, the person being blamed (doctor, financial advisor, etc.) typically says, “Yes, but you said you understood and accepted the risks.”

Complaining and blaming about a loss indicates that although the risk was understood, the true tolerance level of the risk wasn’t understood.

What needs to happen is a candid conversation between the project manager and the stakeholders to fully discuss the risks of a project and how those risks will be handled to cause minimal damage if they materialize. This conversation should be held early in the project before major commitments are made to a specific approach or scope. Unfortunately, in many projects this conversation either happens late in the project or not at all!

How to Assess Risk Tolerance

I am going to suggest my top five questions to help determine risk tolerance on a software project. My approach is to customize these questions to fit your own needs and a variety of audiences. For example, you may have one set of questions for end-users and another set for executive management.

Anyway, here are five good questions to ask stakeholders:

On a scale of 1 to 5, where 1 is very low and 5 is very high:

1.    How comfortable are you with the possibility that your business could not serve customers for 24 hours?
2.    How comfortable are you with customers experiencing problems in doing business with you?
3.    How comfortable are you with your employees being hindered in serving customers?
4.    How comfortable are you in being out of compliance with state and federal laws due to software errors?
5.    How comfortable are you in rushing to meet a deadline at the expense of delivering a correctly working system?

The lower the score, the less tolerant the person is to risk.

Remember, you want to keep the list of questions short and keep the questions simple. Too much complexity and people will ignore them or misinterpret them.

Also, there is no “right or wrong” with the survey results. For example, you may learn that your management is willing to accept high levels of risk. This may explain a lot of things.

Approach #2

Here is another approach using ten questions instead of five. Also, instead of subjective ranking questions, the criteria are more objective. The questions are also more detailed:

1.    What is your overall objective in this project?

2.    How informed are you for projects in which you hold a stake?

3.    How informed would you like to be?

4.    My understanding of business needs is:

5.    If you were told that if installed by the deadline, the project would have some minor to moderate
       problem when installed, and that it may take a week or longer to fix them:

6.    If you were told that if installed by the deadline, the project would have some severe problems when
       installed, and that it may take a week or longer to fix them:

7.    If you were told that there are no contingency or fall-back plans for the installation:

8.    When it comes to understanding software projects, I would consider myself:

9.    How long could the business afford to be without support for the system being developed?

10.    Which of the following best applies to you?

To score this assessment, each “a” = 1, “b” = 2, and so on.  Assign the numeric values to each response and sum them for a total score. The lower the score, the lower the risk tolerance, or the more the person values safety. The higher the score, the more risks they are willing to take.

This survey is just an example and I suggest that you customize it to reflect your own project context and needs.

Conclusion

Determining the project stakeholders’ risk tolerance levels has two key benefits: 1) It tells you if the project team is taking more risk than the stakeholders are comfortable with and 2) it gets people (including the stakeholders) to start thinking about project risks.

After completing the survey, you may find that the stakeholders are going to be much more interested in how much information they receive, how timely the information is, and how much control they have in the implementation decision.

However, a final word of caution here is one more point from Burt, that we ourselves run the risk of losing credibility if we attempt to assess the stakeholders’ risk tolerance and then don’t do anything with it.

Anytime you open the door for stakeholder involvement and then don’t follow through, it can be seen as just a waste of time and effort. If nothing else, use risk tolerance as a point of counsel as you conduct projects. An even better use of risk tolerance is to make it part of the early project initiation discussion.

Bio

Randall Rice is a leading author, speaker and consultant in the field of software testing and software quality. Randy has 30 years experience building and testing mission-critical projects in a variety of environments and is co-author of the book, Surviving the Top Ten Challenges of Software Testing. He is the author and instructor of Testing SOA and Structured User Acceptance Testing courses, presented by Rice Consulting Services. You can contact Randall through his web site at www.riceconsulting.com.