Software Testing Basics: 2009

What is Tetsing?

Sunday, March 29, 2009

Before starting in details, pause for a second and try to answer – What is TESTING?

Is it

A process of demonstrating that errors are not present?
OR
A way of establishing confidence that a program does what it is supposed to do?
OR
A means of achieving an error-free program by finding all errors?

Let’s answer this.

Testing is –

- A process of executing a program with the intent of finding errors.

- A “DESTRUCTIVE”, yet creative process

Now the next question is Why Testing?

Why Testing?

Saturday, March 28, 2009

• To verify that all requirements are implemented correctly (both for positive and negative conditions)

• To identify defects before software deployment

• To help improve quality and reliability.

• To make software predictable in behavior.

• To reduce incompatibility and interoperability issues.

• To help marketability and retention of customers.

We would discuss next Cost of Ineffective Testing?

Cost of Ineffective Testing

Cost of ineffective testing should be understood in below listed three perspective:

Time

• Late Releases.
• Projects need to be reworked or abandoned.

Money

• Budget over-runs.
• Defects are 100 to 1000 times more costly to find and repair after deployment.

Quality

• Uncertainty about the quality of product.
• Products released with undiscovered or unresolved defects.

Before going further, we compare SDLC and STLC to understand where does Testing stand in SDLC.

SDLC vs STLC

SDLC - Software Development Life Cycle

STLC - Software Testing Life Cycle

Below is the pictorial representation how STLC interact with SDLC.

.

Testing Life Cycle (Water Fall Model)

Also, before moving deep into Sofware Testing Fundamentals, it is good to understand Testing Life Cycle with respect to traditional Water Fall Model.

Below pictorial representation should help:

Testing Life Cycle

We saw the STLC interaction with SDLC in previous post. Details of various phase of Test Life Cycle are as follows:

Test Requirements Identification

• Validate for testability

Test Planning and Scenario Design

• Develop Test Objectives
• Identify Test Items
• Resources and Schedules

Test Case Development

• Test Case Specification
• Pre-requisites, post-requisites and Acceptance criteria

Prepare Test Bed

• Test Data Preparation
• Test Environment Setup

Test Execution

• Execute Test cases / Test Scripts and Validate Results
• Defect Reporting
• Defect fixes and retesting

Test Result Analysis

• Defect Analysis
• Determining Test Set Coverage and Effectiveness

Next, we will have a look on Basic Forms of Testing.

Basic Forms of TESTING

There are two basic forms of testing

• Verification Testing
• Validation Testing

Verification

• “Are we building it right?”
• Each step in building the product complies with the process to yield a right
• product.

Validation

• “Are we building the right thing?”
• Determination of the correctness of the products of software development with respect to the user needs and requirements.

Since we understand basic forms of testing, it is a good time to have a look on V-Model.

V - MODEL

The V-Model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape.

The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing.

Representation of V Model is as follows:

The another representation may be as follows:

Before moving deep in to each phase of V Model, let's pause and have a look on Testing Techniques.

Testing Techniques

Testing technique is set of rules and approaches used to test an application or product within acceptable norms.

Various Testing Techniques are:

Static Testing

Dynamic Testing

- White Box Testing
- Black Box Testing

Next, we will have a detail look on Static Testing.

STATIC TESTING or REVIEWS

Testing of an object without execution on a computer id referred as Static Testing or Reviews.

Static Testing Scope

- Requirements Feasibility
- Technical Architecture
- Design
- Program specifications
- Test Plans
- Test Cases and Data
- User Documents

Static Testing Advantages

- Capture defects early, so saves rework cost
- Checklist-based approach
- Focuses on coverage
- Group perspective
- Highest probability of finding defects

Static Testing Disadvantages

- Time-consuming
- Cannot test data dependencies
- High skill levels required

Next is Dynamic Testing.

DYNAMIC TESTING

The process of executing a program or system with the intent of finding errors is referred as Dynamic Testing.

Further, Dynamic Testing is sub divided under this are:

- White Box Testing

- Black Box Testing

WHITE BOX TESTING

The following types of testing fall under the white box testing technique - Unit Testing, Integration Testing etc.

BLACK BOX TESTING

The following types of testing fall under the Black box testing technique - System Testing, Acceptance Testing etc.

Next, we understand difference between White Box and Black Box Testing.

WHITE and BLACK Box Testing

WHITE BOX TESTING

- Structure & Design based Testing
- Program-Logic driven Testing
- Examines the internal structure of program

Following are the advatages of White Box Testing:

- High Code coverage – Exhaustive (thorough) path testing
- Program logic is tested
- Internal boundaries are tested
- Testing is algorithm specific

BLACK BOX TESTING

- Specification based Testing
- Business-Transaction driven Testing
- No concern to internal behavior and program structure

Following are the advatages of Black Box Testing:

- User’s perspective
- Focus on features, not implementation
- Big-picture approach

Next, we will discuss various phases of V MODEL. And the next one is UNIT TESTING

UNIT TESTING

Unit Testing is testing of individual units or groups of related code.

Unit Testing is carried out with Unit Test Plan to verify the following:

• Standards specified (Ex. GUI related)
• Field validations, boundary values
• Basic functionality of the component (add/delete/modify/query activities)
• Code coverage and exception handling
• Negative or destructive testing
• Memory leaks
• Data computation/integrity

White Box Approach for Unit Testing

• Identification of Testable units/modules
• Focus on code and logic
• Design logic-based Test cases and prepare Test data
• Run tests and verify output through path traversing

Black Box Approach for Unit Testing

• Equivalence Partitioning
• Boundary Value Analysis
• Error Guessing

Next, we will discuss Black Box approach for Unit Testing.

Black Box Approach to Unit Testing

There are three commonly used black box techniques for Unit Testing:

• Equivalence Partitioning
• Boundary Value Analysis
• Error Guessing

Equivalence Partitioning

A method of classifying Test Data, in groups based on its attributes. Following steps are involved:

- Identify Equivalence Classes
- Identify Test Cases

For Example - Input data for a program unit usually falls into a number of partition e.g. - "All negative integers, zero, all positive numbers".

Boundary Value Analysis

• Output domain
• User queries
• Numerical data
• Mouse picks on menu
• Output format requests
• Responses to prompts
• Command key input

Error Guessing

Iis is an ad Hoc method to identify tests likely to expose errors based on experience and intuition.

Some areas to guess are :

• Empty or Null lists / strings
• Zero instances / Occurrences
• Blanks or null characters in strings
• Negative numbers

Next is - Integration Testing.

INTEGRATION TESTING

Integration Testing is combining and testing multiple components together.

Features include Integration of modules, programs and functions by

- Testing Internal interfaces for programs
- Testing External interfaces for modules

Next is - Inter, Intra and External Interface.

Intra-Module, Inter-Module and External Interfaces

It is imporatnt to understand the difference between Intra-Module, Inter-Module and External Interfaces.

Let's try the same with following Typical System Diagram:

Inter modular interface - Between units within a module

Intra modular interface - Between modules within a system/application

External interface - External systems/applications

Stubs and Drivers

Stubs and drivers would be required to simulate modules and facilitate interface testing during integration.

Stubs can be as simple as routines that automatically return on a call, or they can be more complicated and return simulated results.

Test drivers provide the facilities needed to execute a program (e.g., input or files, commands).

Test drivers permit generation of data in external form to be entered automatically into the system.

Next is Approach for Testing Interface.

Interface Testing

There are different set of Test Techniques, Test Environment Set up, and Test Approach to test different kind of Intra Module, Inter Module, and External Interface.

Intra Module Interface

Test Technique - White Box
Test Environment - Development Environment
Test Approach - Check for parameter, procedure, database updating etc

Inter Module Interface

Test Technique - Black Box
Test Environment - Isolated Development Environment or CIT Environment
Test Approach - Identify pre and post process to validate interface

External Interface

Test Technique - Black Box
Test Environment - Simulate (Production) Live Environment
Test Approach - Identify interfaces and validate against external system

Next is various Integartion Techniques.

Integration Test Techniques

Integration Testing is done when the different modules of the product have been integrated together.

It is an orderly process, which is carried out until all the modules have been integrated into the system.

Following are the most commonly used techniques:

1. Big-Bang Integration or Non-Incremental

2. Incremental Integration

There are two incremental appraoches as follows:

- Top-Down Integration

- Bottom-Up Integration

Top-Down Technique helps to trap key interface defects earlier but core functionality tested late in cycle. And opposite is true for Bottom-Up Testing.

Next is System Testing.

SYSTEM TESTING

It is also known as - SYSTEM INTEGRATION TESTING or SIT.

System testing tests end to end scenarios.

This can be understood using a simple example of a chain of events of ATM Transaction.

What is SIT?

• A Black-box Testing technique
• Specifications-based Testing
• Testing carried out by non-Development Team
• Simulated environment testing
• Live/Simulated user data
• Tests the whole system
• Functional and non-functional requirements tested
• Business transaction-driven testing
• Compatibility errors uncovered
• Performance limitations uncovered

Next is - Different types of SIT.

Types of System Testing

SIT is THE MOST important phase of SDLC as well as STLC. Applications/System should be ready to go live after SIT.

SIT includes all kinds of possible testing needed for an applications or syatem such as:

• Functional testing (Sanity / Regression )
• Performance and Scalability testing
• Volume Testing
• Usability testing
• Configuration testing
• Installability testing
• Disaster and Recovery testing
• Reliability / Availability testing
• Statistical Testing based on Usage Models

etc.

ACCEPTANCE TESTING

It is also known as User Acceptance Testing or UAT.

Acceptance testing starts, when the System Testing Team has signed off and the System is ready to be tested by it’s intended users.

Mostly UAT is done by CUSTOMERS.

UAT demonstrates that the system meets mutually agreed Acceptance criteria on:

- Critical Requirements
- Minimum Performance level
- Maximum Defect Detection Rate

Two Forms of Acceptance Testing - ALPHA and BEETA Testing.

Alpha Testing is performed by end users within a company but outside development group.

Beta Testing is performed by a sub-set of actual customers outside the company.

Next is When to Stop Testing?

When to Stop Testing?

This may sound weired but it is true we must set the exit point to testing. Some of guidelines are as follows:

• When time runs out
• According to some calculations derived from statistics available from Test cases
• Target test coverage attained
• Certain number of errors found
• Error detection rate drops below a threshold
• Predicted density of errors left drops below a threshold

Pros and Cons of St to Stopping Testing

+VE
- Testers fail to find the defects for long hours
- Defects found are too minor/cosmetic in nature

-VE
- Error levels are on a higher side compared to the expectations Ex. stoppers and critical bugs
- Too many errors found in a short duration of testing
- Some components fail. till they are corrected testing cannot progress.
- Key components are missing.
- Test environment is unstable

Next would be Test Artifacts.

TEST ARTIFACTS

As we have now basic idea of Testing and Test Engineering. It is a good to understand what are the Test Artifacts required in a typical STLC.

Here is a list:

• Test Strategy
• Test Plan
• Test Case
• Test Data
• Requirement Traceability matrix
• Test Coverage Matrix
• Test Scripts
• Test Log
• Defect Report
• Test Closure Reports

As we cover most of the basics for Manual Testing, it is good if we can now have a basic understaing of Test Automation.

TEST AUTOMATION

Test Automation is new emerging industry because of its cost effectiveness and its success rate. However, one must understand that test automation can not be used everywhere.

Below are some guidelines:

Preferred for:
• Complex and time-consuming tests
• Tests requiring a great deal of precision
• Tests involving many simple, repetitive tests
• Tests involving many data combinations

Not preferred for:
• One-time only tests
• Testing peripheral devices
• Subjective assessment tests (look and feel based)

Simple Vs Sophisticated Automation

It is also a very important to evalute the degree of sophistication needed for automation. Below graphical representation should help to understand how cost and automation related:

.

Testing Tools

In last post we, were talking about Test Automativon, it is a good idea to have a look on various types of Testing Tools too.

Client Server Test Tools

Web Application Test Tools

Network & Security Test Tools

Subscribe to: Posts (Atom)