Black Box Testing – LO4
Black Box Testing – LO4
One factor that makes software systems difficult to test is the complexity of the system. An important contributing factor to software complexity is the underlying complexity of the business rules being modeled and implemented.
The term business rule is used here in a generic sense to mean the logic that determines the way decisions are made. By looking at the software, the business rules would generally be reflected in the code by the number of if statements or case statements. Note however, since black-box techniques are being investigated there is no need to access thecode.
These decision criteria and actions that have been labeled as business rules are typically expressed in the requirements and specification for the software. Requirements documents encode these rules in various ways: as text, as UML use cases, or sometimes in tabular form.
Decision tables are a precise yet compact way to model complex rule sets and their corresponding actions [wiki]. If such a table is given in the requirements documents, thenit can be used to generate test cases. If it is not, then one can be developed (in consultation with the clients who know the business rules). This approach is particularly useful when there is a complex set of inter-related conditions that determine the actions. (For simple business rules, this technique may not be necessary.)
How to make a Decision Table?
Let’s consider a simple situation first and then add some complexity in a later example. A decision table consists of two portions. The top rows specify the various combinations of conditions; the bottom rows indicate the actions that result.
For a simple example, let’sconsider the task of an admissions office of a college. This office needs to determine which students to admit and which ones to recommend for scholarships. They base these decisions on many factors. To keep our initial example simple, let’s suppose that we have rated applicants as having a high ACT or SAT score, or a low score; and that some applicants are rated as having a compelling admissions essay. These ratings are determined by the admissions staff and are inputs to our system.
Thus, there are three (3) conditions. In the decision table, all possible combinations of these conditions should be covered; there should be 23 = 8 true/false columns in the table.
|Condition1: High ACT/SAT score||T||T||T||T||F||F||F||F|
|Condition2: Low ACT/SAT score||T||T||F||F||T||T||F||F|
|Condition3: Compelling admissions essay||T||F||T||F||T||F||T||F|
|Action #1: Admit||x||x||x||x|
|Action #2: Give scholarship||x||x|
|Action #3: Do not admit||x|
|Action #4: Not possible||x||x||X|
Now for the actions: a decision can be made to admit the student or to deny admission; also a decision to recommend a scholarship. The decision about which action to take in each situation is one that is determined by the admissions staff. We are just trying to model their strategy.
Thus, the strategy represented by this decision table is to admit a student with a compelling admissions essay even though that student has low test scores. (This is the column that has conditions F,T,T.) You will also notice that the last row is labeled “Action #4:Not possible”. This is used to represent the fact that, for instance, it is not possible for one student to have both high test scores and not to have high test scores.
Generating Test Cases
Now that there is a decision table, how are test cases created? The general guideline is that each column determines one test case. Thus, 8 test cases can be developed from this decision table
|High test scores, low test scores, compelling essay||Illegal input||Testing of high and low test scores which is not possible|
|High test scores, low test scores, no compelling essay||Illegal input||Testing of high and low test scores which is not possible|
|High test scores, compelling essay||Admit and give scholarship||Covers the case of a good student who wrote a strong essay|
|High test scores, no compelling essay||Admit and give scholarship||Covers the case of a good student|
|Low test scores, compelling essay||Admit||Covers the case of a student with low test scores but good essay|
|Low test scores||Deny||Low test scores and not a strong essay|
|Compelling essay||Admit||Covers the case of a student with average test scores but good essay|
|Not high scores, not low scores, no compelling essay||Illegal input||This case is not clear. We would need to ask the clients what this really means|
Now this example can be expanded to a somewhat more realistic example. Here is a list of ten possible criteria that an admissions office could use in determining whom to admit and to whom to award scholarships. This is followed by a more detailed set of possible actions, including early admission and two levels of scholarships to award.
- Condition #1: high gpa(high school gpa)
- Condition #2: medium gpa
- Condition #3: low gpa
- Condition #4: high ACT or SAT
- Condition #5: medium ACT or SAT
- Condition #6: low ACT or SAT
- Condition #7: compelling admissions essay
- Condition #8: relative of an important alumnus/alumna
- Condition #9: has important abilities (sports, music, theater, etc.)
- Condition #10: first generation college
- Action #1: admit early
- Action #2: admit
- Action #3: give scholarship
- Action #4: give high scholarship
- Action #5: deny admission
Constructing Decision Table
Since there are 10 conditions, the full decision table would consist of 210 columns. It might be argued that this is too large of a table to be practical. However, please note that the complexity is in the decision to be made, not in this analysis. Because of the complexity, there are many test cases to consider. Of course, the analysis may find that some of these are not possible combinations. As has been shown in the simpler version, the combination of two or three of condition #1, #2, and #3 is not feasible.
Rather than trying to complete the 1024 columns, let’s make the original example a little more complex by considering four (4) conditions. Consider the decision table below.
|Condition #1: high school gpa||T||T||T||T||T||T||T||T||F||F||F||F||F||F||F||F|
|Condition #4: high ACT or SAT||F||F||F||F||T||T||T||T||F||F||F||F||T||T||T||T|
|Condition #9: has important abilities (sports, music, theater, …)||F||F||T||T||F||F||T||T||F||F||T||T||F||F||T||T|
|Condition #10: first generation college||F||T||F||T||F||T||F||T||F||T||F||T||F||T||F||T|
|Action #1: admit early||x||x||x||x||x|
|Action #2: admit||x||x||x||x||x||x||x||x||x||x|
|Action #3: give scholarship||x||x||x||x|
|Action #4: give high scholarship||x||x||x||x||x||x|
|Action #5: deny admission||x||x|
1. How many test cases will we need to cover these decisions?
a. 5 – one for each row of actions
b. 4 – one for each row of conditions
c. 16 – one for each column of conditions
d. We cannot tell how many
2. Consider the column with the label #2 in the decision table. What is a test case for this set of conditions?
a. Input: highgpa, first generation college; output: admit, give high scholarship
b. Input: highgpa, first generation college; output: admit early
c. Input: admit, give high scholarship; output: highgpa, first generation college
d. Input: high ACT or SAT, Has important abilities; output: admit, give high scholarship
These black-box testing techniques – input space partitioning (equivalence class partitioning), boundary value analysis, and decision tables – can work together in testing. The decisions of the decision table can help to determine the partitions. The details of these partitions can be used to help to identify the boundary values.
Let’s revisit the previous student college admission example. The decision table can be used to give us three (3) equivalence classes about ACT/SAT test scores: low, medium, and high. Similarly, we have three (3) classes for high school gpa: low, medium, and high. The input about the compelling admissions essay, having an alumni relative, having particular abilities (sports, music), and being first-generation college each have two equivalence classes: those students who do have these, and those who don’t. For these last four characteristics, boundary analysis does not add anything.
However, for the characteristics of ACT/SAT test scores and high school gpa, boundary analysis can be helpful. For ACT scores, let’s arbitrarily say that 32 and above are high scores, score between 27 and 31 (inclusive) are medium level, and low scores are those 26 and below. For high school gpa, let’s set high to be 3.5 and above, medium to be 2.5 to 3.5, and low to be below 2.5.
You do not have access to this data
You do not have access to this data
Dorothy Graham, Erik Van Veenendaal, Isabel Evans, Rex Black. 2007. Foundation of Software Testing, ISTQB Certification. Cengage Learning products, London, UK.