mirror of
https://github.com/boostorg/test.git
synced 2026-01-22 05:42:35 +00:00
194 lines
12 KiB
HTML
194 lines
12 KiB
HTML
<HTML>
|
||
<HEAD>
|
||
<TITLE>Boost Test Library generic recommendations</TITLE>
|
||
<LINK rel="stylesheet" type="text/css" href="../style/btl.css" media="screen">
|
||
<LINK rel="stylesheet" type="text/css" href="../style/btl-print.css" media="print">
|
||
<META http-equiv="Content-Language" content="en-us">
|
||
<META http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
||
</HEAD>
|
||
<BODY>
|
||
<DIV class="header"> <A href="../index.html">Boost.Test</A> > <A href="Tutorials.html"> Tutorials </A> > <SPAN class="current_article"> Introduction
|
||
into testing</SPAN></DIV>
|
||
<DIV class="body"> <IMG src="../btl1.gif" width="252" height="43" alt="Boost Test logo">
|
||
<H1>Introduction into testing</H1>
|
||
<P class="first-line-indented">For almost everyone, the first introduction
|
||
to the craft of programming is a version of the simple "Hello World" program.
|
||
In C++, this first example might be written as</P>
|
||
<PRE class="code">#<SPAN class="reserv-word">include</SPAN> <<SPAN class="literal">ostream</SPAN>>
|
||
|
||
<SPAN class="cpp-type">int</SPAN> main()
|
||
{
|
||
std::cout << <SPAN class="literal">“Hello World\n”</SPAN>;
|
||
}
|
||
</PRE>
|
||
<P>This is a good introduction for several reasons. One is that the program
|
||
is short enough, and the logic of its execution simple enough that direct
|
||
inspection can show whether it is correct in all use cases known to the new
|
||
student programmer. If this were the complexity of all programming, there
|
||
would be no need to test anything before using it. In programming as a new
|
||
student experiences it, testing is pointless and adds unneeded complexity.</P>
|
||
<P class="first-line-indented">However, no actual programs are as simple as
|
||
an introductory lesson makes “Hello World” seem. Not even “Hello World.” In
|
||
all real programs, there are decisions to be made and multiple paths of execution
|
||
based on these decisions. These decisions could be based on user input, streaming
|
||
data, resource availability and dozens of other factors. The programmer strives
|
||
to control the inputs, and results of these decisions, but no one can keep
|
||
all of them clearly in mind once the size of the project exceeds just a few
|
||
hundred lines. Even “Hello World” hides complexities of this sort in the
|
||
simple seeming call to std::cout.</P>
|
||
<P class="first-line-indented">Since the individual programmer can no longer
|
||
determine the correctness of the program, there is a need for a different
|
||
approach. An obvious possibility is testing the program after construction.
|
||
Someone develops a set of test cases, where inputs are given to the program
|
||
such that the behavior and outputs of a correctly performing program are
|
||
known. The performance of the new program is compared to known standards
|
||
and the new program either passes or fails. If it fails, attempts are made
|
||
to fix it. If the test cases are carefully chosen, the specifics of the failure
|
||
give an indication of what in the program needs to be fixed.</P>
|
||
<P class="first-line-indented">This is an improvement over just not knowing
|
||
whether the program is working properly, but it isn’t a big improvement.
|
||
If the whole program is tested at once, it is nearly impossible to develop
|
||
test cases that clearly indicate what the failure is. The system is too complex,
|
||
and the programmer still needs to understand almost all of the possible outcomes
|
||
to be able to develop tests. As always, when a problem is too big and complicated
|
||
a good idea is to try splitting it into smaller and simpler pieces.</P>
|
||
<P class="first-line-indented">This approach leads to a layered system of testing,
|
||
that is similar to the layered approach to original development and should
|
||
be integrated into it. When writing a program, the design is factored into
|
||
small units that are conceptually and structurally easier to grasp. A standard
|
||
rule for this is that one unit performs one job or embodies one concept.
|
||
These simple units are composed into larger and more complicated algorithms
|
||
by passing needed information into a unit and receiving the desired result
|
||
out of it. The units are integrated to perform the whole task. Testing should
|
||
reflect this structure of development.</P>
|
||
<P class="first-line-indented">The simplest layer is Unit Testing. A unit is
|
||
the smallest conceptually whole segment of the program. Examples of basic
|
||
units might be a single class or a single function. For each unit, the tester
|
||
(who may or may not be the programmer) attempts to determine what states
|
||
the unit can encounter while executing as part of the program. These states
|
||
include determining the range of appropriate inputs to the unit, determining
|
||
the range of possible inappropriate inputs, and recognizing any ways the
|
||
state of the rest of the program might affect execution in this unit.</P>
|
||
<P class="first-line-indented">With so many general statements, an example
|
||
will help clarify. Imagine the following procedural function is part of a
|
||
program, and the programmer wants to test it. For the sake of brevity, header
|
||
includes and namespace qualifiers have been suppressed. </P>
|
||
<PRE class="code"><SPAN class="cpp-type">double</SPAN> find_root( <SPAN class="cpp-type">double</SPAN> (*f)(<SPAN class="cpp-type">double</SPAN>),
|
||
<SPAN class="cpp-type">double</SPAN> low_guess,
|
||
<SPAN class="cpp-type">double</SPAN> high_guess,
|
||
std::<SPAN class="cpp-type">vector</SPAN><<SPAN class="cpp-type">double</SPAN>>& steps,
|
||
<SPAN class="cpp-type">double</SPAN> tolerance )
|
||
{
|
||
<SPAN class="cpp-type">double</SPAN> solution;
|
||
<SPAN class="cpp-type">bool </SPAN> converged = <SPAN class="literal">false</SPAN>;
|
||
|
||
<SPAN class="reserv-word">while</SPAN>(not converged)
|
||
{
|
||
<SPAN class="cpp-type">double</SPAN> temp = (low_guess + high_guess) / <SPAN class="literal">2.0</SPAN>;
|
||
steps.push_back( temp );
|
||
|
||
<SPAN class="cpp-type">double</SPAN> f_temp = f(temp);
|
||
<SPAN class="cpp-type">double</SPAN> f_low = f(low_guess);
|
||
|
||
<SPAN class="reserv-word">if</SPAN>(abs(f_temp) < tolerance)
|
||
{
|
||
solution = temp;
|
||
converged = true;
|
||
}
|
||
<SPAN class="reserv-word">else </SPAN><SPAN class="reserv-word">if</SPAN>(f_temp / abs(f_temp) == f_low / abs(f_low))
|
||
{
|
||
low_guess = temp;
|
||
converged = false;
|
||
}
|
||
<SPAN class="reserv-word">else</SPAN>
|
||
{
|
||
high_guess = temp;
|
||
converged = false;
|
||
}
|
||
}
|
||
|
||
<SPAN class="reserv-word">return</SPAN> solution;
|
||
}
|
||
</PRE>
|
||
<P class="first-line-indented">This code, although brief and simple is getting
|
||
long enough that it takes attention to find what is done and why. It is no
|
||
longer obvious at a glance what the intent of the program is, so careful
|
||
naming must be used to carry that intent.</P>
|
||
<P class="first-line-indented">Thanks to the control structures, there are
|
||
some obvious execution paths in the code. However, there are also a few less
|
||
obvious paths. For example, if the root finder takes many steps to converge
|
||
to an acceptable answer, the vector that is holding the history of steps
|
||
taken may need to reallocate for additional space. In this case, there are
|
||
many hidden steps in the single push_back command. These steps also include
|
||
the chance of failure, since that is always a possibility in a memory allocation. </P>
|
||
<P class="first-line-indented">A second example notes that the value of the
|
||
function at the low guess has not been tested, so there is the chance of
|
||
a zero division. Also, if the value of the function at the high guess is
|
||
zero, the root finder will miss that root entirely. It may even fall into
|
||
an infinite loop if no root lies between the low and high values.</P>
|
||
<P class="first-line-indented">In this unit, proper testing includes checking
|
||
the behavior in each possibility. It also includes checking the function
|
||
by giving inputs where the correct answer is known and checking the results
|
||
against that answer. Thus, the unit is tested in every execution path to
|
||
assure proper behavior.</P>
|
||
<P class="first-line-indented">Test cases are chosen to expose as many errors
|
||
as possible. A defining characteristic of a good test case is that the programmer
|
||
knows what the unit should do if it is functioning properly. Test cases should
|
||
be generated to exercise each available execution path. For the above snippet,
|
||
this includes the obvious and the not so obvious paths. Every path should
|
||
be tested, since every path is a possible outcome of program execution.</P>
|
||
<P class="first-line-indented">Thus, to write a good testing suite, the tester
|
||
must know the structure of the code. The most dependable way to accomplish
|
||
this is if the original programmer writes tests as part of creating the code.
|
||
In fact, it is advisable that the tests are produced before the code is written,
|
||
and updated whenever structure decisions are changed. This way, the tests
|
||
are written with a view toward how the unit should perform instead of reproducing
|
||
the programmer’s thinking from writing the code. While black box testing
|
||
is also useful, it is important that someone who knows the design decisions
|
||
made and the rationale for those decisions test the code unit. A programmer
|
||
who can’t devise good tests for a unit does not yet know the problem at hand
|
||
well enough to program dependably.</P>
|
||
<P class="first-line-indented">When a unit is completed and tested, it is ready
|
||
for integration with other units in the program. This is integration should
|
||
also be tested. At this point, the test cases focus on the interaction between
|
||
the units. Tests are designed to exercise each way the units can affect each
|
||
other.</P>
|
||
<P class="first-line-indented">This is the point in development where proper
|
||
unit testing really shines. If each unit is doing what it should be doing
|
||
and not creating unexpected side effects, any issues in testing a set of
|
||
integrated units must come from how they are passing information. Thus, the
|
||
nearly intractable problem of finding an error while many units interact
|
||
becomes the less intimidating problem of finding the breakdown in communications.</P>
|
||
<P class="first-line-indented">At each layer of increasing complexity, new
|
||
tests are run, and if the prior tests of the components are well designed
|
||
and all issues are fixed, new errors are isolated to the integration. This
|
||
process continues, in parallel with development, from the smallest units
|
||
to the completed program.</P>
|
||
<P class="first-line-indented">This shows that there is a need to be able to
|
||
check and test code snippets such as individual functions and classes independent
|
||
of the program of which they will become a part. That is, the need for a
|
||
means to provide predetermined inputs to the unit and to check the outputs
|
||
against expected results. Such a system must allow for both normal operation
|
||
and error conditions, and allow the programmer to produce a thorough description
|
||
of the results.</P>
|
||
<P class="first-line-indented">This is the goal and rationale for all unit
|
||
testing, and supporting testing of this sort is the purpose of the Boost.Test
|
||
library. As is shown below, Boost.Test provides a well-integrated set of
|
||
tools to support this testing effort throughout the programming and maintenance cycles
|
||
of software development.</P>
|
||
</DIV>
|
||
<DIV class="footer">
|
||
<DIV class="footer-body">
|
||
<P> © <A name="Copyright">Copyright</A> <A href='mailto:jphillip at capital dot edu (please unobscure)'>John
|
||
R. Phillips</A> 2006. <BR>
|
||
Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||
file <A href="../../../../LICENSE_1_0.txt">LICENSE_1_0.txt</A> or copy
|
||
at <A href="http://www.boost.org/LICENSE_1_0.txt">www.boost.org/LICENSE_1_0.txt</A>)</P>
|
||
<P>Revised:
|
||
<!-- #BeginDate format:Sw1 -->18 February, 2006<!-- #EndDate -->
|
||
</P>
|
||
</DIV>
|
||
</DIV>
|
||
</BODY>
|
||
</HTML>
|