How Are We Doing - Part 1

Abstract

Because of the widespread awareness of the Software Engineering Institute's Capability Maturity Model (CMM) and its related upgrades over the past several years, it seems that many organizations tend to view their status based a five-level scale.

Although I like the CMM and think it has many good points, I also know that many organizations that build software will never go up the CMM ladder. The questions that come to my mind are,

"Does it make sense to assess an organization to a model they will likely never embrace?"

"How can these organizations determine quickly and objectively how they are doing and where they can improve?"

"How can an organization improve in the short-term and sustain those efforts without using extensive models or frameworks?"

Introduction

This week as I was checking out of the hotel, I noticed a simple little card in the room that read at the top, "How Are We Doing?" It was a chance for me as a customer to simply rate the overall service I experienced during a certain point in time, in a certain room of the hotel. Unlike other survey cards, which sometimes have so many questions I just give up, this one only had three questions:

1. Are we meeting your expectations? (Yes/No) Why?

2. What else can we do to make it your place?

3. Is there anyone in particular that made your stay with us more enjoyable?

Then, it had a place to give my room number, date, and name.

As I read the card and thought about the many things that must happen in a hotel to provide a comfortable experience for a guest, the simplicity of the response card impressed me. This simple card let me tell the hotel everything they needed to know about what might have gone wrong and how they could fix it. It caused me to think about why people make things so complex in Information Technology quality improvement.

This article is in two parts. This first part discusses the nature of organizations, models and assessments. The second part will discuss how to objectively and quickly determine how your IT organization is performing and how to show short-term and sustained results.

The Basis of Research

All of my research for this article was performed in the practice of performing assessments as a independent consultant in a wide variety of organizations throughout the United States and Canada over the past eight years. These organizations include those in healthcare, state and federal governments, insurance, transportation, defense, banking, retail, and manufacturing.

 The Nature of Organizations

To understand why not every organization will benefit from assessments based on standard frameworks, we must first understand the nature of organizations.

Organizations are a collection of people

These people may or may not use technology. They may or may not care about the quality of their work. The foremost thing to remember about assessments is that they are people-centered. Your assessment may address processes, technology and any other number of areas. There will human factors to each of those areas. People have a major impact on assessments because they provide information that must be analyzed and understood in the context of the organization. People can provide biased information and perhaps conceal information in the assessment. The assessor must be able to apply a "smell test" of reading between what people say and what they really mean, to determine the validity of people's input.

Organizations have shared practices

There will be a collection of processes performed in an organization, whether or not they are documented. Sometimes I find it more accurate to call how an organization does its work as "practices." These practices may be performed inconsistently, yet they determine how the organization fulfills its mission. These practices are shared among the people in an organization, which means that there will be multiple perspectives of how they are being performed.

Organizations have complex, yet undocumented, hierarchies that people are expected to understand and respect

These are the political aspects of an organization that people learn by experience in working in the organization. It is important to understand the political nature of an organization, as it impacts how, why, and when people do their work.

Organizations often do not communicate well

 It is rare to find the organization that takes the time and concerted effort to communicate within itself. People need time to talk outside of the context of a meeting! Then, think about how your organization communicates with your customers.

Organizations may or may not function as healthy teams

Teamwork is such a fine quality that most groups simply do not reach the measure of a true team. People may work together to reach a goal or do a job, but how effective are they and how do they treat each other along the way?

Organizations often have unique business or other external factors that determine how they fulfill their mission

Each organization has a set of external concerns that must be considered in how it performs. However, it is amazing how an assessor can see many points in common across organizations in all industries. For example, as an assessor of many organizations both large and small, I have observed that the larger the organization, the more likely it is they are out of control when it comes to internal processes.

The view is always brighter from the top of the organization

Senior management tends to see the bright spots and minimize the many trouble points that cause frustration with workers and customers. However, it's the little things that make a difference to customers and employees. Unfortunately, many senior managers never get an objective view of what is going on at the lower levels of the company. People are either afraid to state their honest opinion or middle management covers up the real situation to avoid getting in trouble. Here's a test for senior management: Go to a pay phone, call your company and ask for yourself. See how you are treated when people don't know who you are. That's the view that your customers have of your company.

The Nature of Models

Perhaps the most basic definition we could give to a model is that it is a simplified view of the real world. To be applicable to wide audience, the model must achieve two things:

1. Be within the scope of experience of the real world, not a theoretical one

2. Be a subset of the real world, as not everyone will have the same concerns. Because of this, we must conclude that all models will have gaps in a particular user's experience. We can also conclude that there will be things we can learn from models.

Any simplification of the real world can be seen as a model. Since the real world can be seen from a variety of views, depending on your own frame of reference, you may or may not find a lot in common with some models. In other cases, you may find that a particular model makes a lot of sense, but the leadership in your organization will not buy in to it.

Models simplify the real world

To understand the nature of complex things, we need to be able to visualize them in a smaller, simpler way. We don't get all of the details, just enough to gain an initial understanding. When I was a teenager, I used to enjoy building model cars and planes. I know adults who still enjoy modeling and do an amazing job at building small replicas of all sorts of things. Although I might build a model of a car and see where the engine goes, I don't build the detailed view of the engine – just a little piece of plastic that looks like the engine. The same goes for our view of the real world in a model – there are many aspects that we would need to drill deeper to fully understand. By the way, you can buy a model of an engine if you really want to understand how one works!

Models are a subset of the collection of many people's experiences

 You may experience and understand how to do something one way and I might have a totally different view. A model attempts to consider a variety of viewpoints.

Models help people find common ground in knowing how they are doing

People in organizations often want to know how they are performing in relationship to other organizations. Almost like a grading scale used in school, people like to have a scorecard. Models help provide a basis for the scoring by showing what's important. For example, in school our teachers had to know what makes a good reader so they would know how to grade us. They determined that a good reader needs to understand words, comprehend the meaning, and achieve a certain speed in reading. This gives teachers everywhere a common ground for judging how well a person reads. Likewise, models help us understand how to judge how we are doing based on activities of others.

Models are not a methodology or process to be followed

 It is critical that we understand that a model describes "what" is to be done, not "how" it is done. The process must be determined outside of the model, but yet must be consistent with the intent and framework of the model. For example, if you have adopted the CMM, you need to design or find a process that you can repeat across projects in your organization. You need to define measurements for your organization that will tell you where to improve. However, the CMM does not tell you how to do your work or the specific things in your organization to measure.

Models are for understanding, not necessarily for transformation

People often expect the magic of organizational transformation to be in the model. You can hear this in statements like, "When we get to level 3 our overall software quality levels will be higher." In reality, reaching a level in any model just indicates that you are doing the things that will position you to see the results you desire. It's the people, both in management and in the trenches that will transform the organization to achieve higher levels of performance.

To be absolutely clear on this, I will say again that I am not against models, by any means. In fact, as technology and collective experience changes, new models may be needed. My point is that models have limitations and not every organization will be able to find usefulness in a given model. Furthermore, models are only a part of the solution to which some organizations have not even correctly determined the problem!

The Nature of Assessments

An Assessment Is A Snapshot

An assessment is a picture taken at one point in time of one or more aspects of an organization's performance. Today's snapshot may be different from last year's or next year's.

Assessments Are Performed From A Variety Of Perspectives

These perspective include:

• Self-assessments,
• Internal objective assessments (such as performed by a consultant) and
• External assessments (such as performed by customers).

Assessments Are Often Focused In One Or More Areas

These include assessments for risk, readiness, and process. An assessment can also be performed to determine compliance to a model or framework, such as the CMM. Assessments Tell You Where You Are

One of the great benefits of an assessment is that it gives you a frame of reference. To improve a current situation, you need to know three things:

• Where you are at currently
• Where you want to be at some point in the future
• How you plan to reach your goal I think of an assessment like the map at a shopping mall .

To me, the map does little good if there isn't a red dot or "X" that reads "You are here." With the marker, I can tell where I'm at and how far away I am from my goal. 

Assessments Alone Do Not Motivate Change

Another analogy is that of a medical exam. The examination tells my doctor things like blood pressure, weight, cholesterol level and heart rate. From that information, the doctor can make recommendations to me about risk factors in heart disease and other illnesses that I want to avoid in the future. He can also warn me about current problems that need immediate attention. However, it is up to me to make the changes. If I make changes to my diet and lifestyle and maintain those changes, then the exam was useful. Otherwise, the exam was just information and I'm still at risk as before.

Likewise, the motivation to take action from an assessment must come from within yourself or from within the organization.

Where Do We Go From Here?

The assessment can also be the basis for building a plan to reach the goal, or at least the next step. We will discuss the plan for improvement in Part 2 of this article. We will also show a method for simply and effectively assessing your performance that makes sense for your organization.