Testing Wireless Applications

NOTE: This is a legacy article. Updated content coming soon.

Abstract

Traditionally, testers have been behind the development curve in defining and adopting new test strategies. This article is a strategy-level overview which describes the background and future of wireless technology, the major risks and critical success factors, an overview of the technologies used to deliver wireless applications, and a strategy for testing wireless applications.

Wireless Technology is More Than the Wireless Web

People are using wireless technologies to build long-range wireless networks and perform short-range personal wireless functions. Some of these wireless applications are extremely critical in terms of risk. For example, mobile applications allow medical professionals to view an EKG chart and other diagnostic data using pagers. Wireless scanners help stores keep track of inventory levels.

Regardless of the application and specific technology, wireless applications require a unique testing strategy that can be identified and documented. In addition, although the future of the wireless web is hard to predict in terms of when and how it will actually take off, it is time to start thinking about the implications of testing wireless applications. Even if the wireless web is very slow to gain stride, many businesses will be building and deploying other types of wireless initiatives to serve customers better and more efficiently.

Some examples of other types of wireless applications beside the wireless web and M-commerce include:

• Handheld devices for applications such as inventory control and shipping
• Pagers for instant messaging and data delivery - even for applications such as medical diagnostics
• PDA applications to sync with notebooks computers and to perform functions such as automated hotel check-in and check-out while in the hotel
• Peer-to-peer applications for workgroup collaboration and gaming
• Wireless networks to allow maximum flexibility in workspace configuration

How Wireless Applications are Developed

 To develop a test strategy for wireless applications, we need to know how they are developed in terms of tools and technologies. We don't have time in this article to cover all technologies, but some of the major ones include:

Wireless Access Protocol (WAP) - An initiative that was started by Unwired Planet, Motorola, Nokia, and Ericsson to define a secure specification that allows users to access information instantly via handheld wireless devices such as mobile phones, pagers, two-way radios, smartphones and communicators.

Bluetooth - A short-range (up to 10m) 2.4 GHz radio technology aimed at simplifying communications among Net devices and between devices and the Internet. It also aims to simplify data synchronization between Net devices and other computers. Products with Bluetooth technology must be qualified and pass interoperability testing by the Bluetooth Special Interest Group prior to release. The Bluetooth 1.0 specification consists of two documents: the Foundation Core, which provides design specifications, and the Foundation Profile, which provides interoperability guidelines.

XML - Extensible Markup Language - allows designers to create their own customized tags, enabling the definition, transmission, validation, and interpretation of data between applications and between organizations.

WML - Wireless Markup Language is an XML language used to specify content and user interface for WAP devices. WML is supported by almost every mobile phone browser around the world. WML pages are requested and served in the same way as HDML pages.

WMLScript - Scripting language used with WML to allow interactivity on WML applications.

It is important to understand that just like web applications are often built using a combination of technologies such as XML, Java, and Java Script, so are wireless applications. The mix depends on the application objectives, project objectives and business strategies.

Wireless application developers use development tool kits provided by the wireless device vendors, such as Nokia, Ericcson and Motorolla. These development kits also contain simulators that allow developers and testers to test wireless applications in a controlled environment.

The Big Picture - How M-Commerce Applications are Delivered

The above figure shows how the M-commerce user conducts a transaction. One of the major differences between this model and the E-commerce model of recent years is the prevalent use of a mobile wallet to facilitate fast and secure payments. Like the E-commerce model, integration is the key to making the business side of M-commerce work.

Primary Concerns and Risks of Wireless Applications

A very useful starting point for a test strategy is to examine the risks. For wireless applications, the risks are:

Correctness - Like any other application, if the information is not correct, people will abandon the application. 

Usability - Wireless applications shrink the user interface to the extent that two concerns emerge greatly when dealing with wireless usability.

1. Interface - The display size of wireless devices is often very small. This affects readability and feature sets in a big way.

2. Ergonomics - Wireless devices are compact and have smaller keys for input. If you have small fingers, the issue is lessened, but if you have fingers that resemble small sausages, you need another way to input data beside your fingers.

Security

Even though WAP, WEP and other security protocols are designed to be a secure technology when used correctly, there is a big difference between knowing and doing when it comes to security. 

In addition to the street threats, there are also the security issues of what happens to the transaction at the gateways, which are controlled by nameless, faceless third parties. Peer-to-peer computing also holds risks for the privacy of data sent between devices.

Performance

Performance issues can result from a variety of sources including:

Slow infrastructure

One of the most pressing challenges in wireless applications is the growing demand on bandwidth. This demand grows daily as new users are coming onto the web and as businesses increase the quantity and quality of media content, such as streaming media.

There is considerable discussion as to whether or not third generation (3G) networks will ever become a reality in the U.S. In addition to slow networks, the reliability of wireless connections leave much to be desired. It's one thing to be able to decipher a garbled voice message, but another to transmit a stream of data across the same faulty connection.

Client bottlenecks

Performance problems can also be traced back to client bottlenecks in WAP applications as well as hardware limitations with the wireless device.

Integration

As in any robust application, wireless or otherwise, integration often plays a major role in the ability of the application to deliver correct and efficient results. Wireless integration concerns include:

• Online wallets for fast and secure online payments
• Business systems which must handle the transactions generated by wireless devices
• Other devices which receive data from wireless devices:
  • Notebooks, which may sync with PDAs
  • Vending machines, which my provide inventory control data to an central business system
  • Handheld units, which may transmit inventory or shipping data to central systems
  • Web-enabled phones, which my send data to other cell phones such as with instant messaging

Flexibility

People want to make the most of their investment in wireless technology and are often cautious about hardware and applications that may become obsolete in a year or less. Therefore, flexibility is a very important issue to wireless users. Flexibility in this context can include:

• Ease of upgrades to software and hardware
• Application flexibility with various hardware and wireless operating systems.

Profitability

This is an opportunity to apply some of the lessons learned from the recent dot-com boom and bust experience. Wireless providers must be able to prove that the envisioned users will actually buy the applications and services under development. In addition, the wireless development companies must be able to survive until the wireless market can support the new technology and pray that a new technology does not catch on and render their product or service obsolete.

Some questions that should be asked about any project include:

• Does the application solve a problem effectively and easily?
• Is there a customer base?
• Can development costs be recouped quickly?

A Testing Strategy for Testing Wireless Applications

Now that we have examined the basis of wireless applications and the risks of wireless technology, let's look at a strategy for testing wireless applications, regardless of the method of application.

The major items to consider are:

• Type of Wireless Application
• Audience Demographics
• Technologies Used
• Tools
• Test Execution Strategies

Type of Wireless Application

Some possibilities include:

• WAP
• M-Commerce
• Workgroup
• Personal productivity
• Peer-to-peer
• Business system integration
• Entertainment

The type of application will determine the type of tests you will need to perform.

Audience Demographics

This will dictate the types of testers you will need to get a representative assessment of usability.

• Age (The most likely acceptance group for wireless apps are teens!)
• Geography - This applies to issues of localization as mobile applications are deployed in a variety of countries.
• Web comfort level - This will determine the important of usability factors.

Technologies Used

This will determine the type of test environment and tools you will need.

• WAP
• Bluetooth, etc.
• XML, WML
• Security protocols

Tools

• WAP Simulators by hardware vendor: Nokia, Ericsson, Motorola
• Capture/Playback - such as TestPartner by Compuware
• Performance Testing - such as QA Load Compuware

Test Execution Strategies

Pilot tests

• Other countries - With early deployment in other countries, it is possible to introduce a product to a smaller and more remote audience. You can practice in a small room!
• Limited user base - This type of test would include usability testing with a small number of users to get an assessment of initial usability.

What Does This Mean for Testing Wireless Apps?

 It's a small world after all!

User interfaces tend to be much smaller than other applications, which translates into smaller test cases.

There's a large part of testing wireless apps that is like other testing.

This includes tests for validating the integration of transactions, platform portability and integration testing, and having a basis for testing such as use cases and/or requirements.

Wireless test strategies can take many forms

Just a few of the many flavors include M-commerce, data collection and delivery, entertainment and workgroup collaboration.

Tools can help

This is especially true in regression and load testing.

The test lab is greatly different than the real world.

Simulators are great, but you need to test also with real people in real-world conditions.

Summary - Dos and Dont's of Testing Applications

The world of testing wireless applications is growing and changing daily. I'm sure that within a few weeks, I will learn even more considerations for this rapidly changing technology. In the meantime, here are my closing thoughts.

Do:

• Ask questions early about application feasibility
• Remember that wireless technology is still very young
• Perform early usability testing
• Be prepared for an onslaught of new development when people and the technology are ready.

Don't:

• Forget that you still need something to base tests upon, such as use cases and requirements.
• Forget that much of M-commerce is based on integration between organizations.
• Forget your audience.

I hope this article helps in your search for information on testing wireless applications. I would like to hear about your challenges and success stories. You can e-mail me from the contact page.