28 Apr Black Box techniques

In this article I’ll explain some of the black box testing techniques that I applied during the testing projects I was involved into: equivalence partitioning, boundary value analysis and state transitioning testing.

What is black-box testing?

Black-box testing, also called behavioral testing, focuses on the functional requirements of the software. Black-box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements of a program. Black-box testing is not an alternative to white-box techniques. Rather, it is a complementary approach that is likely to uncover a different class of errors than white-box methods.

What is equivalence partitioning?

Equivalence partitioning is a black-box testing method that divides the input domain of a program into classes of data from which test cases can be derived. Test case design for equivalence partitioning is based on an evaluation of equivalence classes for an input condition. An equivalence class represents a set of valid or invalid states for input conditions. Typically, an input condition is either a specific numeric value, a range of values, a set of related values, or a boolean condition.

What is boundary value analysis?

Boundary value analysis extends equivalence partitioning by focusing on data at the “edges” of an equivalence class. A greater number of errors tend to occur at the boundaries of the input domain rather than in the “center”. It is for this reason that boundary value analysis (BVA) has been developed as a testing technique. Boundary value analysis leads to a selection of test cases that exercise bounding values. Boundary value analysis is a test case design technique that complements equivalence partitioning.

Example 1: Testing of a numeric text box

This is a very common scenario. We all deal with testing the input fields of our applications. Let’s presume that the text box we are testing accepts values between 1 and 1000. How do we test this using equivalence partitioning and boundary value analysis? Having this specification, we can now set the valid and invalid partitions. Values between infinite minus and 0 represent the first partition. Values between 1 and 1000 represent the second partition and values between 1001 and infinite represent the third partition. The first and the third partition are invalid because they contain all the values that are not accepted in the text box. The second partition is the valid one because it contains all the valid values. For better understanding please see the visual representation below:

Behavior at the edge of each equivalence partition is more likely to be incorrect than behavior within the partition, so boundaries are an area where testing is likely to yield defects. The maximum and minimum values of a partition are its boundary values. A boundary value for a valid partition is a valid boundary value; the boundary of an invalid partition is an invalid boundary value. Tests can be designed to cover both valid and invalid boundary values. When designing test cases a test for each boundary value is chosen. In order to test this situation we must choose the boundary values and create test cases as below:

Input value    Expected result

0    Invalid

1    Valid

1000    Valid

1001    Invalid

This example shows how boundary value analysis and equivalent partitioning analysis work together. It’s a case of looking for limits and then restricting the number of tests to a manageable set. This example was incredibly simple; most applications have numerous complex boundary conditions. While testing the applications within my project I have to consider boundaries such as printing reports using volumes of data. Boundaries can be exposed in print buffers or boundaries in exporting data where an application allows a larger entry in a field than can be exported in a CSV formatted file. There can be many undefined and unrealized boundaries until a defect is found exposing new limitations. And equivalence partitioning can help me narrow the number of tests I need to execute in order to expose boundaries.

Example 2: Testing of a login form

This is another example for using these techniques. Let’s assume that we have a login form with Username and Password fields having the following characteristics:

• Username should have between 10 and 15 characters
• Password should have between 8 and 16 characters.
• Password should have minimum one special character and minimum one upper case letter.

We will begin by creating the partitioning for the username and the password. Given the initial characteristics, we can create the following partitions and decide which one is valid and which one is invalid:

 

The Username is much simpler to test because it does not have so many restrictions. Like in the example above, we can create the table with the boundaries:

Length of the Username	Expected result
9 char – Username1	Invalid
10 char – Username12	Valid
15 char – Username1234567	Valid
16 char – Username12345678	Invalid

 

Judging by the data displayed in the table we can clearly distinguish the four cases that can be tested for the username. Moving on to the password, we will have to take into consideration a lot more test cases because of the various restrictions imposed. First of all we can distinguish the three partitions that result from the number of characters accepted when generating a password: 1. < 8 2. Between 8 and 16 3. > 16 It is clear by now that only the second partition is the valid one and if that the password has less than 8 characters or more than 16 it will never be valid. Given the fact that we have a lot of possible test cases we can greatly reduce their number by only using the valid partition. With this in mind we move on to the next characteristics of the password. As we said before, we must have at least one special character (ex. “ or @) and at least one upper case letter. In consequence we have the following possibilities:

1. No special characters
2. One special character
3. Two special characters
4. No upper case letter
5. One upper case letter
6. Two upper case letters

 

The following table shows the possible cases and their validity:

Test Cases	Valid	Invalid
2.4.7		x
2.4.8		x
2.4.9		x
2.5.7		x
2.5.8	x
2.5.9	x
2.6.7		x
2.6.8	x
2.6.9	x

 

According to the table we obtained by judging the characteristics of the password we obtained four valid test cases and 5 invalid ones. By using the equivalence partitioning and boundary value analysis techniques, we were able to greatly reduce the number of test cases to a vital minimum, decide which test cases are valid and which are invalid thus reducing the time spent testing this functionality.

What is state transition testing?

State transition testing is used where some aspect of the system can be described in what is called a ‘finite state machine’. This simply means that the system can be in a (finite) number of different states, and the transitions from one state to another are determined by the rules of the ‘machine’. This is the model on which the system and the tests are based.

A state table shows the relationship between the states and inputs, and can highlight possible transitions that are invalid.

Tests can be designed to cover a typical sequence of states, to cover every state, to exercise every transition, to exercise specific sequences of transitions or to test invalid transitions.

Example 3 – Testing the login function of a program

Let’s take as an example the testing of a login function, which is defined as in the state diagram below :

 

When testing this scenario I think of four major aspects of the state transition technique:

• The states that the software may be in (open/closed, valid/invalid)
• The transition from one state to another
• The events that cause a transition
• The actions that result from a transition (either an error message is displayed or the user is redirected to the main menu page)

In any given state, one event can cause only one action, but that the same event – from a different state – may cause a different action and a different end state. For example, if you log in with a given password (ex. Password1), you may be logged into the system. Later you may make exactly the same request but the application may refuse to log you into the system because the password was changed by another user. Even though this diagram is not that complex, it still gives enough information for me to design some useful tests:

• The first scenario is the one where the correct username and password is entered the first time;
• The second test case would be the one in which the password is entered incorrectly three times and the account is blocked and the admin is notified;
• A third scenario would be the one in which the correct password is introduced the second time or the third time;

Each state shown in the diagram can be a test condition and depending on the complexity of the log in function we can come up with a large number of test cases for this function alone.

	Insert password	Valid Password	Invalid Password
S1. Wait for password	–	S5	S2
S2. 1st try invalid	–	S5	S3
S3. 2nd try invalid	–	S5	S4
S4. 3rd try invalid	–	–	S6
S5. Access account	–	?	?
S6. Block account	S1 (for new password)	–	–

 

The initial state is of course S1, the Start state, in which the system is waiting for a password. After the password is introduced, depending on the validity, the system shifts from one state to another. Every time the password is valid the system will shift in S5. When the password is invalid the system will shift through the three states in which the user is prompted to retry entering the valid password. (S2, S3, S4). Conclusion The use of these techniques allows me to save time and energy.

Given the fact that I usually have to execute a large number of various test cases in a short period of time, I find these techniques very useful because they help me cover all the angles of the testing scenarios and give me the confidence that my work is done correctly.