Testing Dirty Systems Workshop
3 Day Workshop
Level: Intermediate to Advanced
Dirty systems are those that are complex, undocumented and have unknown defect levels and architecture. These are challenging systems to maintain and test because over time they grow old and fragile.
In this course, based on the book by Randall W. Rice and William E. Perry of the same title, you will learn a process for how to gain understanding of a dirty system, plan and design effective tests, and ultimately improve or clean the system based on what is learned in testing.
Testing Dirty Systems is based on a six-step process that takes from software discovery through test planning and design, test execution, test analysis and reporting, and finally, clean-up. You will leave with knowledge of how tools can help you understand and test dirty systems and how to deal with special testing challenges in dirty systems.
Who Should Attend
• Test analysts
• Testers
• Developers
• Test Managers
• Development Managers
Return on Investment
• Learn how to improve dirty systems and reduce the total cost of maintenance and the risk of failures
• Learn how to find costly and embarrassing problems before your customers find them.
• Understand the key issues in testing dirty systems.
• Learn how to design tests that adequately cover undocumented requirements and business events.
• Advance your career by reinforcing your testing expertise.
Topics
Module TDSA – Introduction to Testing Dirty Systems (.5 hr.)
• What is a “Dirty System?”
• The Dirty System Challenge
• General Testing Concerns
• Application Testing Concerns
• Operational Testing Concerns
Module TDSB – A Process for Testing Dirty Systems (1 hr.)
• Why Have a Process for Testing Dirty Systems?
• Defining Work Processes
• How to Reduce Defects
• The Big Picture of Continuous Improvement
• What is a Process “Under Control”?
• Why Statistics?
• Special and Common Causes of Variation
• The Need to Improve Stable Systems
• Tampering
• A Process for Testing Dirty Systems
• How to Apply the Six-Step Process in Your Organization
-
- Organize Your Test Team
- Prepare for the Work Sessions
- Customize the Six-Step Process as Necessary
- Train Others to Use the Process
Module TDSC - Dirty System Diagnostics (2 hrs.)
• The Black Holes of System Knowledge
• A Process for Analyzing Dirty Systems
• Scanning the System and Documentation
• Identifying Helpful Resources
• Documenting the System’s Change History
• Obtaining the Necessary Resources
• Conducting Exploratory Walkthroughs or Reviews
• Obtaining Interface Information
• Documenting Your Findings
• Diagnostic Analysis Deliverables
• Software Diagnostic Tools
• Software Measurements
o Complexity Measures
• Frequently Asked Questions
• Step Guidelines
• Module Summary
• Case Study - The Mega Bank Case Study
o Analysis of the Mega Bank Case Study
o Understanding Overly Complex Logic
o Relating Functionality to Requirements
o Absence of Defined Requirements
o Incorrect or Vague Requirements
o Missing, Incomplete, or Obsolete Systems Documentation
o Handling Large Size and Scope of Functionality
o Non-Supported Vendor-Developed Software
o Inadequate User Training
o Creating Plans to Accommodate Growth And Integration Needs
o Developing an Adequate Security Policy
o Diagnostic Analysis of the Mega Bank Case Study Application
Module TDSD - Defining a Test Strategy for Dirty Systems (1 hr.)
• What is a Test Strategy?
• What's in a Test Strategy for Dirty Systems?
• Sample Test Strategy
• Test Strategy Checklist
• Module Summary
Module TDSE - Test Planning for Dirty Systems (2 hrs.)
• Defining the Scope and Objectives of Testing
• Mega Bank Case Study Test Objectives
• Test Objectives Checklist
• Identifying System Functions and Attributes
• Gathering Structural and Functional Information
• Obtaining Structural Information
• Obtaining Functional Information
• Identifying Functions to be Tested
o Sample Function List
o Sample Test/Function Matrix
o Determining Evaluation Criteria
o Functional Test Checklist
• Identify Attributes to be Tested
o Describing the Attributes
o Determining Evaluation Criteria
o Prioritizing the Attributes
o Relating the Attributes
• Sample Attribute List
• Attribute Test Checklist
• Creating the High-level Test Plan
o Pulling Together the Pieces
o What is "High-Level"?
o The Purpose of a High-Level Test Plan
o What's in the Plan?
o Things to Remember About Test Plans
o High-level Test Plan Checklist
o Reviewing the Test Plan
• Approving and Finalizing the Test Plan
• Big Picture and Small Picture Views of Test Planning
• Frequently Asked Questions
• Test Planning Guidelines
• Summary
Module TDSF - Problem and Impact Analysis (1 hr.)
• The Impact of Changes in Dirty Systems
• What is Impact Analysis?
• Performing Impact Analysis
• Deliverables
• Usage Guidelines
• Sample Problem Statement
• Sample Impact Analysis Report
• Impact Analysis Checklist
• Summary
Module TDSG - Performing a Risk Assessment (1 hr.)
• Why is Risk Assessment Important?
• Risk Perspectives
• Input for Risk Assessments
• A Risk Assessment Process
• Performing a Project Risk Assessment
• Performing a Business Risk Assessment
• Performing a Functional Risk Assessment
• Sample Risk Assessment Report
• Summary
Module TDSH - Designing a Dirty System Test Environment (1 hr.)
• The Role of Test Environments in Dirty Systems
• Levels of Test Environments and their Effectiveness
• Test Environment Design
• Case Study
• Dedicated Test Environments
• Shared Test Environments
• PC and Server Test Environments
• Virtual Test Environments
• Dealing with Data
• Test Environment Design Checklist
• Summary
Module TDSI - Designing Detailed Tests (2 hrs.)
• How Detailed Tests are Designed
• When to Start
• Where to Start
• A Process for Creating Detailed Tests
• Documenting Test Cases
• Sample Test Cases
• How Tools Can Help in Designing Test Cases
• Types of Test Cases
• The Role of Regression Testing in Dirty Systems
• Test Case Design Checklist
• Test Scripts
o Sample Test Script
o Helpful Tips for Manual Test Scripting
o When Scripts May Not Be Helpful
• Relating and Organizing Tests
o The Requirement/Test Matrix
o The Test/Function Matrix
o Sample Test/Function Matrix
o The Test/Sub-Function Matrix
o The Test/Cycle Matrix
• Summary
Module TDSJ - Test Preparation for Testing Dirty Systems (.5 hr.)
• Preparing Your Testing Process
• Walkthroughs and Reviews
• Trial Tests
• Getting the Test Environment in Place
• Conduct Tester Orientation
• Summary
Module TDSK - Performing the Test (2 hrs.)
• Challenges and Solutions
• Factors in Test Execution
• Managing Dirty Systems Test Execution
• Managing Testware
• The Human Impact on Testware
• A Process for Managing and Controlling Testware
• People Issues in Testing Dirty Systems
o Managing People
o The Top Ten Challenges of Software Testing
o Other Human Issues in Software Testing
• Managing Dirty System Test Execution
• Key Test Execution Tasks for Dirty Systems
• A Test Execution Process
• People Issues in Test Execution
• Test Execution Tools
• Tracking Defects
• People Issues in Defect Tracking
• Defect Tracking Tools
• Tracking Test Results
• Responsibilities in Test Tracking
• Test Tracking Tools
• Controlling The Test Environment
o The Role of Configuration Management
o A Process for Test Environment Control
o Responsibilities in Test Environment Control
o Test Environment Control Tools
o Managing Change in Testing Dirty Systems
o Responsibilities for Change Management
• Managing Expectations in Dirty Systems
o The Importance of Managing Expectations
o Responsibilities in Expectation Management
• Summary
Module TDSL - Dirty System Test Execution by Test Phases (1 hr.)
• Phases of Testing
o Unit Testing
o Integration Testing
o System Testing
o User Acceptance Testing
• Dirty System Test Execution by Test Types
• Summary
Module TDSM - Manual vs. Automated Testing of Dirty Systems (1 hr.)
• Advantages and Disadvantages of Manual Testing
• What’s Needed for Manual Testing?
• Manual Test Organization
• Automated Testing
o Automation Pre-Requisites
o Automated Test Organization
• More About Test Tools
• Summary
Module TDSN - Analyzing Dirty System Tests (1 hr.)
• Observe, Document, and Analyze Test Results
• Reviewing the Risk Assessment
• Maintaining the Baseline of Test Results
• Test Analysis Deliverables
• Test Analysis Tools
• Test Analysis Metrics
o Recording Test Results
o Test Coverage Standards
o Code Coverage
o Test Case Coverage
• Frequently Asked Questions
• Guidelines for Success
• Summary
Module TDSO - Test Reporting in Dirty Systems (1 hr.)
• Defect Reporting in Dirty Systems
o How Defects Can be Reported
o The Role of the Defect Administrator
o What is Reported?
o Characteristics of Good Defect Reports
o How This Information Can be Used
• Test Summary Reporting in Dirty Systems
o Preparing the Summary Report
o Reporting on the Efficiency and Effectiveness of Dirty System Testing
o What to Consider When Creating Test Summary Reports
o Adding Value
o Testing Summary Report Template
o Objective of the Test Summary Report
o Example of Test Summary Report
o How to Interpret the Report
• Summary
Module TDSP - Cleaning Dirty Systems (1 hr.)
• The Difference Between Quality of Design and Quality of Performance
• What’s Needed?
• A Process for Cleaning Dirty Systems
• Deliverables From the Clean-Up Process
• Clean-up Checklist
• Techniques for Cleaning Dirty Systems
o Software Reviews
o Re-engineering Code Based on Standards and Good Practice
o Extending and Enhancing Data Validation
o Improving the Database Architecture
o Extending and Expanding Documentation
o Building and Extending Testware and Test Data
o Identifying and Correcting Security and Control Deficiencies
• Assessing the Need for Cleaning
• Guidelines for Success
• Summary