Testing COTS Applications

2 Days

Register for the Online Version

Take the Online Demo

"Anytime I can take something away from a course that I can use in my day to day, work projects is a good class." - Gina Boster

"Randy is very knowledgeable regarding testing in general.  I recommend him as an instructor to anyone interested in testing." - Elijah Hayden

This course is designed to teach the process of planning, performing and evaluating the tests of COTS applications in a way that customers can easily learn and apply.

"COTS" is a term used to describe "commercial off-the-shelf" software. COTS applications are becoming increasingly more popular for solving mission and business objectives because the development effort is performed by the vendor of the product instead of a customer's internal staff. However, the customer and users of the product have a huge responsibility to ensure that the application will work correctly in the operational environment after it is integrated.

COTS applications can be seen in single standalone implementations or integrated with other applications to form a COTS-based implementation. Some of the challenges this course will address are:

• How to design tests for an application that does not have defined requirements available

• How to design tests that validate the support of the organization’s mission processes

• How to evaluate the completeness of the testing effort in terms of test coverage

• How to estimate the testing effort if you don’t have time to count function points or lines of code

• How to test the integration of a COTS application with other COTS or in-house products, including the "glue" that an organization develops for custom integration efforts

• How to test COTS customization and vendor extensions

• How to test user manuals and operating procedures provided by the vendor

• How to design tests that can deal with future product versions

Return on Investment 

• Find gaps in support of your operational processes by COTS application.
• Have an increased level of confidence in your new COTS application before it is implemented for production use.
• Validate that the vendor has fulfilled delivery requirements.
• Add a new perspective of testing knowledge to your existing skill set.
• Learn how to design tests that adequately cover requirements and business events.
• Advance your career by reinforcing your testing expertise.

Who Will Benefit

• Test Managers
• Test Team Leaders
• Test Engineers
• Test Designers and Analysts
• Testers
• End-users
• Software Developers

To get the most from this course, attendees should have a basic level of computer literacy and have the ability to adapt general processes to their own specific applications. Prior testing experience and skills are helpful, but not required for this course.

 Program Information

This course is presented on an in-house basis only unless offered as a special public course. Contact us for information about how to bring this course into your organization.

Content and Structure

Topics

Module 1 - COTS Applications Defined (1 Hr.)

This module lays a foundation of understanding about COTS applications and some of the key terminology associated with COTS applications. A case study is presented that will serve as the basis for discussion and exercises throughout the course.

• What is COTS?
• Variations of COTS - COTS-based, GOTS, MOTS
• What is the Purpose of COTS?
• Why are the Challenges of COTS Applications?
• What are the Challenges of Testing COTS Applications?
• What are the Risks of Implementing COTS Applications?
• Key Terminology
• An Introduction to the COTS Lifecycle
• Roles and Responsibilities for COTSCase Study in COTS Testing
  • Vendors
  • Customers
  • Senior Management
  • Users
  • In-house Developers
  • Test Team Leadership
  • Testers
  • QA Analysts and Leaders
• Additional Resources

Module 2 - A COTS Testing Framework (.5 Hr.)

This module presents the COTS testing framework that will be the driving process for this course.

• A COTS Testing Framework
• The COTS Testing Framework Explained
• How the COTS Testing Framework Fits Into the Overall COTS Lifecycle
• The Evolutionary Nature of the COTS Lifecycle and How it Impacts Testing
  
  

Module 3 - Developing a COTS Testing Strategy (1.5 hrs.)

This module presents how to develop a test strategy for COTS applications and how to use the strategy to start planning very early in a project what you will need to test it.

• Objectives of the Strategy
• COTS Risks and How They Drive the Test Strategy
• Anticipating New Risks that are Identified During the Project
• Identifying the Critical Success Factors for COTS Applications
• COTS Integration Strategies and How They Impact the Testing Strategy
• Determining the Uniqueness of Your Project
• How to Create a COTS Test StrategyExercise
  • The Elements
  • The Process
  • An Example Standard
    
    

Module 4 - Planning the Test of a COTS application (3 Hrs.)

This module presents a complete planning process for testing COTS applications.

• Process Overview
• A Mapping of Test Planning Activities to the COTS LifecycleRequired Input and Entry Criteria
  • Acquisition
  • Integration
  • Customization
  • Testing
  • Implementation
• What Needs to Be Verified
• What Needs to be Validated
• The COTS Test Planning Process Step-by-step
• Review a Sample COTS Test Plan
• Detailed COTS Test Planning
• Detailed Test Planning Techniques and Deliverables
  • Test Cases
  • Test Scenarios
  • Test Scripts
  • Test Descriptions
  • A Detailed Test Planning Framework
• Adding Traceability of Tests to:Types of Test Matrices
  • Customer needs and desires
  • Acceptance criteria
  • Test objectives
• Detailed Test Planning by Example
• Testing Integration "Glue"
  
  

Module 5 - Performing the Test of a COTS Application (2.5 Hrs.)

This module presents how to prepare and perform the testing of COTS applications, including integration and regression testing. As part of this module, we will perform the tests designed in the previous module on a COTS application.

• Step 1 - Build the Test EnvironmentStep 2 - Orient and Train Testers
  • Hardware
  • Test Data
  • Software
  • Tools
  • Processes
  • Verify Correct Replication Required to Test Customizations and Vendor
  • Extensions
  • Verify Integration Feasibility
• Step 3 - Perform TestsStep 4 - Evaluate Test Results
  • Verify environment
  • Perform tests
  • Take backups
• Step 5 - Record Defects
• Step 6 - Perform Regression Testing
• Ongoing Test Environment Management
  
  

Module 6 - Analyzing and Reporting the Results of COTS testing (1.5 Hrs.)

In this module, you will learn how to interpret the results of testing COTS applications and how to use that information to implement the product and to improve the COTS acquisition, customization, integration, testing, and implementation processes.

• Categorizing Defects
• Writing Defect Reports
• Writing Status Reports
• Writing Final Test Reports
• Making Sense of the Information to Improve ProcessesExercise
  • Requirements Definition
  • Acquisition
  • Integration
  • Customization
  • Testing
  • Implementation
    
    

Module 7 - Human Factors in COTS Testing (.5 Hr.)

Human factors are very important in testing COTS applications and this module describes some of the most critical issues to be prepared to deal with.

• Vendor Interaction
• Getting Information for Test Planning
• Getting Support for Testing
• Getting Cooperation for Integration Testing
• Getting Customer and User Input and Assistance in Testing
  
  

Resources

• Checklists and Templates
• Glossary
• Bibliography

Deliverables

• Course notebook with slides images, worksheets, checklists, complete examples and supporting text
• You will have the basic information needed to perform effective COTS testing in your organization.