Software Construction: Unit Testing

A flaw in a component or in any part of the software that causes the system to behave incorrectly.

It is about the human action that produces the incorrect result.

Making sure that our system delivers what the user needs it to deliver (are we building the right system?)

Making sure that our system delivers what the specification wants it to deliver, that it is free of bugs (are we building the system right?).

Tge use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions.

Benefits

• Reduces costs
• Reduces human error
• Reduces variance in test quality from different individuals
• Reduces the cost of regression testing

The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met.

Software Testability is dominated by two problems

• How hard is it to provide the test values to the software?
• How hard is it to observe the results of test execution?

⇒ How hard is it to find faults in the software?

How easy is it to provide a program with the needed inputs, in terms of values, operations and behaviors?

Input from hardware sensors or distributed software is harder to provide.

How easy is it to observe the behavior of a program in terms of its outputs, effects on the environment and other hardware and software components?

Software that affects hardware devices, databases or remote files has low observability.

A test case is a multipart artifact with a definite structure. Containing test case values, prefix values, postfix values and expected results necessary for execution and evaluation.

Test case values: The input values needed to complete an execution of the software under test.

Expected results: The result that will be produced by the test if the software behaves as expected.

Test oracle: It uses the expected results to decide whether a test passed or failed.

Prefix values: Input necessary to put the software into the state to receive the test case values.

Postfix values: Input needed to be sent to the software after the test case values are sent.

A set of test cases.

A test case that is prepared in a form to be executed automatically on the tested software and produces a report.

A set of assumptions, concepts and tools that support test automation.

JUnit can be used to test...

• an entire object
• parts of an object (a method or some interacting methods)
• the interaction between serveral objects

It is primarily intended for unit & integration testing, not system testing!

A JUnit test class contains...

• one or more test methods (collection of test methods)
• Methods to set up the state before and update the state after each test and before & after all the tests (test methods marked with @Before / @After).

Each test method checks a condition (assertion) and reports to the test runner whether the test failed or succeeded.

All of the methods return void.

JUnit method name: @Test public void testSomething(){ /*testing stuff */}

Example assertions:

• assertTrue(boolean)
• assertTrue(String, boolean)
• fail(String)

A test fixture is the state of the test

• Objects & variables that are used by more than one test
• Initialzations (prefix values)
• Reset values (postfix values)

Different tests can use the objects without sharing the state. Objects used in test fixtures are declared as instance variables.

They should be initialized in a @Before method and can be deallocated or reset in an @After method.

public class Min

{

public static <T extends Comparable<? super T>> T min (List<? extends T> list){

if (list.size() == 0){
throw new IllegalArgumentException ("Min.min");
}

Iterator<? extends T> itr = list.iterator();
T result = itr.next();

if (result == null) throw new NullPointerException ("Min.min");

while (itr.hasNext()){ // throws NPE, CCE as needed
T comp = itr.next();
if (comp.compareTo (result) < 0) result = comp;
}
return result;

}

}

IllegalArgumentException (Using an empty list is not legal for Min.min()!)

These tests validate whether the tested method works with legal input.

Tests with exceptions

• null list (list is null)
• null element with multiple elements (list containing multiplle elements and at least one null element)
• null single elements (list only containing a single null element)
• incomparable types (e.g. list containing strings & int)
• empty elements (empty list)

Tests without exceptions

• single element
• two elements

Simple example: Adding two numbers

• Adding a given pair of numbers (integers) is just like adding any other pair. Therefore ou really only want to write one test.

Data-driven unit tests call a constructor for each set of test values

• Same tests are then run on each set of data values. The collection of data values defined by method tagged with @Parameters.

Example constructor (of the DataDrivenCalcTest class): public DataDrivenCalcTest (int v1, int v2, int expected){}

Example parameter class: @Parameters public static Collection<Object[]> parameters(){} 

Unit tests can have actual parameters (normal @Test methods don't hava parameters!). These classes are tagged with @Theory.

Contract Model: Assume, Act, Assert

• Assumptions (preconditions) limit values appropriately
• Action performs activity under scrutiny
• Assertions (postconditions) check result

Data for parametrized unit tests comes from @DataPoints methods.

DataPoints used for example:

@DataPoints
public static String[] animals = {"ant", "bat", "cat"};

@DataPoints
public static Set[] animalSets = {
   new HashSet (Arrays.asList ("ant", "bat")),
   new HashSet (Arrays.asList (“bat", “cat", “dog“, “elk”)),
   new HashSet (Arrays.asList (“Snap”, “Crackle”, “Pop"))
};

All combinations of values from @DataPoints annotated methods where assume clause is true (4 of 9 combinations in example).

@DataPoints format is an array.

By automating as much as possible.

Test frameworks provide very simple ways to automate our tests. It is however no silver bullet, as it doesn't solve the hard problem of testing: What test values to use?

This is test design, the purpose of test criteria.