How Are We Doing - Part 2
In Part 1 of this article, I discussed the Capability Maturity Model (CMM) and how prevalent it has become in assessing an organization's performance. This is good news in terms of raising awareness of the importance of processes. However, for some organizations that will never go above level 1, it makes me wonder if it makes sense to measure them against the five-level CMM.
We discussed the nature of organizations, the nature of assessments, and the nature of models in part 1. I related the example of a simple hotel survey that read "How are we Doing?" at the top of the survey card. Although assessing your IT organization's performance may be more involved than assessing a hotel's performance, there are some points in common. The main point is that the survey does not have to the involved to be effective.
The Assessment Process
Self-assessment is Not Enough
There are two views to this assessment - Internal (Our View of How We are Doing) and External (Our Customers' and Users' View of How We are Doing). It is important to consider both views as we may think we're doing great, but those we serve may have a different perception. This assessment can also identify where the perceptions differ.
While self-assessments can help us identify strengths and weaknesses in our performance, the self-view is inherently biased, even if performed by an independent third party. An independent assessment is more objective than one performed by the organization, however, the assessor(s) are at some points dependent on the information they obtain from within the organization.
Points of Measurement
In this informal assessment, there are more internal than external measures. This is because we want to keep the assessment short and simple for the customer, but also because there are many things the customer can't rate, such as problem solving.
Keep in mind that this assessment is a sample starting point. You can add or delete criteria as appropriate to your organization's needs. The overall objective is for the assessment to be simple, especially for the customer.
Internal Measures
Responsiveness
- Timeliness
- Delivering what was requested
Teamwork
- Cooperation
Communication
- Lines
- Accessibility
- Common Terminology
- Directions
Problem Solving
- Definition
Learning
- Measurement
- Reflection/Review
Practices
-
Consistency
-
Awareness
Competency
- Management
- Team
Leadership
- Vision
- Influence
- Commitment to Goals
- Commitment to Quality
- Commitment to Customers
External Measures
Responsiveness
- Timeliness
- Delivering what was requested
Communication
- Lines
- Accessibility
Teamwork
- Cooperation with customer
Competency
- Management
- Team
Practices
- Consistency
Designing the Survey
Simplicity is the goal, so with that objective in mind, here is how I have created a sample survey.
How Are We Doing?
An Internal Performance Survey
|
Score 1 = poor, 5 = wow! |
Comments | |
ResponsivenessWas the project delivered when promised? |
1 2 3 4 5 |
|
| Did the project deliver what was requested? | 1 2 3 4 5 | |
TeamworkDid the project team exhibit a spirit of cooperation? |
1 2 3 4 5 |
|
CommunicationWere the lines of communication understood by the project team? |
1 2 3 4 5 |
|
| Were people accessible to speak with during the project? | 1 2 3 4 5 | |
| Was there a common set of terminology used? | 1 2 3 4 5 | |
| Did communication occur both to and from people? | 1 2 3 4 5 | |
Problem SolvingHow well was the problem initially defined? |
1 2 3 4 5 |
|
LearningWere key items measured on the project, such as time and defects? |
1 2 3 4 5 |
|
| Was there a set-aside time for people to reflect and review how the project was performed? | 1 2 3 4 5 | |
PracticesDid the project follow a method consistent with other projects performed in this organization? |
1 2 3 4 5 |
|
| Did the project team show an awareness of the value of consistency with other projects? | 1 2 3 4 5 | |
CompetencyDid management demonstrate competency in running the project? |
1 2 3 4 5 |
|
| Did the project team demonstrate competency in performing the project? | 1 2 3 4 5 | |
LeadershipDid the leadership on the project clearly communicate their vision of what was to be accomplished? |
1 2 3 4 5 |
|
| Did project leadership exhibit the ability to influence people? | 1 2 3 4 5 | |
| Did project leadership show a commitment to achieving project goals? | 1 2 3 4 5 | |
| Did project leadership show a commitment to delivering a quality project? | 1 2 3 4 5 | |
| Did project leadership show a commitment to meeting customers' needs? | 1 2 3 4 5 |
Did anyone in particular on the project exhibit an attitude that made a positive difference in the outcome of the project? If so, who?
Did anyone in particular in the customer area exhibit an attitude that made a positive difference in the outcome of the project? If so, who?
What one thing would you suggest doing differently on the next project?
How Are We Doing?
Customer Survey
|
Score 1 = poor, 5 = wow! |
Comments | |
ResponsivenessWas the project delivered when promised? |
1 2 3 4 5 |
|
| Did the project deliver what was requested? | 1 2 3 4 5 | |
TeamworkDid the project team exhibit a spirit of cooperation? |
1 2 3 4 5 |
|
CommunicationWere people accessible to speak with during the project? |
1 2 3 4 5 |
|
| Did communication occur both to and from people? | 1 2 3 4 5 | |
Problem SolvingHow well was the problem initially defined? |
1 2 3 4 5 |
|
LearningWas there a set-aside time for people to reflect and review how the project was performed? |
1 2 3 4 5 |
|
PracticesDid the project follow a method consistent with other projects performed in this organization? |
1 2 3 4 5 |
|
CompetencyDid management demonstrate competency in running the project? |
1 2 3 4 5 |
|
| Did the project team demonstrate competency in performing the project? | 1 2 3 4 5 | |
LeadershipDid the leadership on the project clearly communicate their vision of what was to be accomplished? |
1 2 3 4 5 |
|
| Did project leadership exhibit the ability to influence people? | 1 2 3 4 5 | |
| Did project leadership show a commitment to achieving project goals? | 1 2 3 4 5 | |
| Did project leadership show a commitment to delivering a quality project? | 1 2 3 4 5 | |
| Did project leadership show a commitment to meeting customers' needs? | 1 2 3 4 5 |
Did anyone in particular on the project exhibit an attitude that made a positive difference in the outcome of the project? If so, who?
Did anyone in particular in the customer area exhibit an attitude that made a positive difference in the outcome of the project? If so, who?
What one thing would you suggest doing differently on the next project?
How to Gather Results
The sample survey questions are written to be answered in the context of a single project. The ideal time to gather this kind of information would be in a project retrospective or post-project review. To track progress, an assessment could be performed at the end of every project.
However, if your organization doesn't work on a project basis, or if you just want to get an overall snapshot of your performance, you can change the wording of the questions to a present tense. I recommend the project assessment, if possible, as it gives people something tangible to think about in answering the questions.
You can distribute the survey to people individually, or have people fill them out during a project review or retrospective.
How to Compile the Results
After you have gathered the survey results, you can use a spreadsheet to record all of the responses. The spreadsheet will give you the capability to average and chart the responses.
How to Make Sense of the Results
For this kind of survey, a simple bar chart is fine to show the results. Each major point of assessment can be a bar (Figure 1). It is easy to show the results of several assessments on a single bar chart (Figure 2). You can also show your goals on the chart.
Figure 1 - The First Assessment Results on a Bar Chart (Baseline)
Your first assessment will be your initial baseline. Your initial baseline might be low, but that's alright. The important thing is to be able to have a way to compare future assessments. I would recommend not looking to other organizations to benchmark with on this assessment because their environments and customers are different than yours.
Figure 2 - Comparison of Assessment Results to a Baseline
A Final Word of Warning
It is possible to go overboard with measurements and this technique is no exception. One thing that really irritates me is to be the respondent of a survey where the surveyor seems more concerned with getting the numbers recorded instead of listening to how I feel about a recent situation. You may find that listing to a single customer complaint reveals more information than this little survey scorecard.
Summary
There are many ways to measure your organization's performance. You may choose the CMM, CMMI, or any number of other frameworks and models. The assessment in this article is to give a way to those who want a simple way to know how they are doing and want to improve.
If you decide to use this assessment, be sure and let me know how it goes for you. You can write me at the contact us page.
- Hits: 3577