Testing in the Cloud
Three-Day Course 
Now available in e-Learning format!
This course teaches how to test services and applications that are cloud-based. You will learn what makes cloud technology unique to testers, and how to best assure the quality of computing you receive from cloud-based applications. The lessons are reinforced with hands-on exercises.
You will gain understanding of cloud technologies and architecture, learn the two major perspectives of cloud testing – the provider and the consumer views, and learn where best to focus your efforts.
Cloud computing raises important questions for testers, such as:
• What can reasonably be tested?
• Where should we focus testing efforts?
• Which testing techniques and tools are most effective?
• Which risks can testing help reduce?
This course helps you answer these questions for your situation.
Program Information
This course is presented on an in-house basis only unless offered as a special public course. Contact us for information about how to bring this course into your organization.
Module 1 - Introduction
• Course Objectives
• What is Cloud Computing?
- The NIST Definition
- Gartner Definition
• Characteristics of Cloud Computing
• Not a Unified View
• Your View of Cloud Computing
• The Key Players
• Interactions Between Cloud Computing Actors
• What Cloud Brokers Do
• Five Main Principles of Cloud Computing
• Benefits
• Risks
• Types of Clouds
- Public Clouds
- Private Clouds
- Community Clouds
- Hybrid Clouds
- Vertical Clouds
• Cloud Reference Model
• Major Service Patterns
- Software as a Service (SaaS)
- Platform as a Service (PaaS)
- Infrastructure as a Service (IaaS)
• An Example
• Example Services Available to a Cloud Consumer
• Major Activities of Cloud Providers
• Other Service Patterns
• Scope of Controls Between Provider and Consumer
• The Role of Virtualization
• Extending SOA
• Cloud Service Management
• The Importance of Testing
• What This Means to Testers
- What Can we Test?
- What Should we Test?
- How Can we Test?
Module 2 - Defining Your Cloud Test Strategy
• Why a Test Strategy?
• Key Test Planning Components
• What to Include
– The Cloud Computing Goals And Strategy For Your Organization
– Purposes of Testing
• Cloud consumer
• To validate business processes
• To verify service levels are met
• Performance, scalability, etc.
• To find defects (secondary)
• To verify security measures are appropriate and effective
• To select the right products and providers
• Cloud provider:
• To verify service levels are met
• Performance, scalability, etc.
• To find defects
• To verify security measures are appropriate and effective
– Scope
– Risks and Critical Success Factors
– Tools and Techniques
– Roles and Responsibilities
– Assumptions and Constraints
– Levels and Types of Testing
– Measurements
• Knowing How Much to Test – The Extent of Testing
• How to Define a Cloud Test Strategy
• Example
• Exercise
Module 3 - Test Environments
• Requirements for Test Environments
• Extending Virtual Environments
• Characteristics of Cloud Test Environments
• The Role of Virtual Test Environments
• OpenStack Initiative
- OpenStack Compute
• Three Major Platforms
• Major Virtual Environment Vendors and Platforms
• Example – Cross-Platform Browser Testing
• Designing Test Environments
• Managing and Controlling Test Environments
• Variations
• Exercise – Designing a Sample Cloud Test Environment
Module 4 - Functional Testing of Cloud Applications
• Examples
• SaaS
• Pros and Cons
• Layers of Cloud Testing
• Views of Testing
• Identifying Functions to Test
• How Functions are Implemented
• Service Implementation
• Prioritizing Functions
• Testing Workflows
• Taking the Organizational View vs. Product View
• Methods
- Data-driven testing
- Interoperability testing
- Usability testing
- Test optimization
• The Relationship Between Process and Data
• Structured Testing
- Top-Down Test Planning & Design
- Structured Test Case Design
- Functional Decomposition with FreeMind
- Minimal Essential Test Strategy (METS)
• Physical Items
• Physical Test Metrics
• Functions
• Metrics
• Time Budgeter
• METS iPhone App
• Agile Testing
- Agile Testing Cycles vs. Traditional Testing Cycles
- The Goals of Agile Testing
- Benefits of Agile Testing
- Characteristics of Agile Testing
- Dealing with Change
- Test-Driven Development (TDD)
- Dealing With the Downsides of Agile Methods
• The Basis for Test Conditions
• Business Process Testing
• Classification Trees Using CTE-XL
• Evaluating Tests
• Characteristics of Good Incident Reports
• Incident Reporting and Tracking - Things You Need to Know
• Sample Incident Categories
• Sample Incident Priorities
• How Can This Data be Used?
• Regression Testing
- What is Regression Testing?
- The Process
- Issues
- How Much is Enough?
- Tip
• Exercise – Design Functional Tests
• Exercise – Perform Functional Tests
Module 5 - The Role of Test Automation
• How Automation Can Help in Cloud Testing
• The Risks
• Which Functions and Tests Can be Automated?• Tool Options
- Commercial Tools
- Open Source Tools
- Free and Cheap Tools
- Scripting
• Exercise – Evaluate Test Tools
Module 6 - Testing Cloud Security
• How Important is Security?
• Cloud Security Advantages
• Security Relevant Cloud Components
• Cloud Security Risks
• Key Security Questions
• The Nature of Security Tests
• Compliance
• What are Your Compliance Requirements?
• Compliance Issues
• Data Security Risks
- Testing Cloud Data Security
- Authentication
- Testing Authentication
- Encryption
• Volume-based Encryption
• Application Level Encryption
• File-based Encryption
• Policy and Key Management
• The Role of SLAs
• On-premises Key Management
- What to Test
• Auditing
• The Threat
• The “Hardware Roof of Trust”
• Governance
• A Process for Cloud Security Testing
• Putting it Together
• Guidelines from NIST
• The Cloud Security Alliance
• Further Resources
• Exercise – Design Security Tests
Module 7 - Testing Cloud Performance
• Examples of Cloud Applications and Performance Concerns
• How We Got Here
• The Performance Impact of Cloud Computing
• Service Level Agreements
- Non-negotiable Agreements
- Negotiable Agreements
• Performance Through the Cloud
• Profiling Workload and Capacity Levels
• Designing Performance Tests
• Understanding Where the Traffic Goes
• Performance Test Tools
• How Cloud Performance Tools Work
• How Cloud-based Performance Tools Work
• Steps in Designing a Cloud Performance Test
• Exercise – Design and Perform a Cloud Security Test
Module 8 - Testing Cloud Reliability and Availability
• What Customers Expect
• The Role of Testing
• The Role of Service Level Agreements (SLAs)
• Key Point About SLAs
• What This Means
• How to Test and Monitor Reliability and Availability
• How to Measure Reliability and Availability
• You Must Decide What is Important
• Traditional Monitoring
• The Challenge: Geo Dispersed Cloud Applications
• Possible Solutions
• Unified Monitoring
• Example Dashboard
• SLA Report
• Exercise – Assess Reliability and Availability Using Monitored Data
Module 9 - Testing Mobile Computing
• Understanding the Marketplace
- Money for Apps?
- Why Mobile Apps?
- Why a “Mobile Cloud”?
• Major Quality Issues in Mobile
• Example – Mobile Application
• Example – Mobile Web Site + Mobile App
• Usability
• Three Categories of Mobile Devices
• Factors for Mobile Computing Quality
• App Testing Functional Correctness
• Mobile Site Functional Testing
• Testing Interoperability
• Testing Device Compatibility
• Testing Mobile Usability
• Diminishing Returns in Usability Testing
• Testing and Monitoring Mobile Performance
• Testing Mobile Security
• OWASP Top 10 Mobile Risks
• Dynamic Analysis
• Static Analysis
• Mobile Security Test Tools
• Mobile Test Tools
• Simulators
• The Role of Test Automation
• Exercise – Designing Mobile Tests
Module 10 - Transitioning to the Cloud
• What to Consider When Transitioning Applications to the Cloud
• Understanding the Business Needs and Value
- Understand the Risks
• The Risk Watch List
• Example: Testing Risk and Return
• Example: Risk Mitigation
- Have a Cloud Transition Strategy
- Understand the Skills and Experience Needed
- Define Governance Methods
- Get the Architecture in Place
- Decide Which Platforms are to be Transitioned
- Understand Which measures define success.
- How Testing Can Help Measure the Success of Cloud Computing
• Exercise – Define a Cloud Transition Plan
Module 11 - Summary
• Top Ten Points
• Questions and Answers