mirror of
https://github.com/boostorg/test.git
synced 2026-01-26 07:02:12 +00:00
152 lines
11 KiB
HTML
152 lines
11 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
|
|
|
|
<HTML>
|
|
<HEAD>
|
|
<TITLE>Test Execution Monitor</TITLE>
|
|
<LINK rel="stylesheet" type="text/css" href="style/btl.css" media="screen">
|
|
<LINK rel="stylesheet" type="text/css" href="style/btl-print.css" media="print">
|
|
<META http-equiv="Content-Language" content="en-us">
|
|
<META http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
</HEAD>
|
|
<BODY>
|
|
<DIV class="centered">
|
|
<TABLE class="body-table" cellspacing="3" >
|
|
<TR>
|
|
<TD id="body"> <A name='TOP'><IMG src='../../../c++boost.gif' width='277' height='86' alt="Boost logo"></A>
|
|
<H1>Boost Test Library: Test Execution Monitor</H1>
|
|
<P class="page-toc"> <A href="#Introduction">Introduction</A><BR>
|
|
<A href="#Benefits">Benefits</A><BR>
|
|
<A href="#Introduction">Example</A><BR>
|
|
<A href="#MonitorCompilation">Compilation</A><BR>
|
|
<A href="#Tests">Test/Example Programs</A><BR>
|
|
<A href="#Rationale">Rationale</A><BR>
|
|
<A href="#Design">Design</A><BR>
|
|
<BR>
|
|
Also see: <A href="unit_test_framework.htm">Unit Test Framework</A> </P>
|
|
<H2><A name="Introduction">Introduction</A></H2>
|
|
<P class="1-line-indented">The Boost Test Library's Test Execution Monitor provides
|
|
a main() function which calls a user-supplied <SPAN class="new-term">test_main()</SPAN>
|
|
function. The library supplied main() relieves users from messy error detection
|
|
and reporting duties. The Test Execution Monitor is intended for fairly simple
|
|
test programs or to dig a problem in an existent production code. <A href="prg_exec_monitor.htm">Program
|
|
Execution Monitor</A> may be more suitable to monitor production (non-test) programs.
|
|
<A href="unit_test_framework.htm">Unit Test Framework</A> may be more suitable
|
|
for complex test programs. </P>
|
|
<H2><A name="Benefits">Benefits</A></H2>
|
|
<P class="1-line-indented">The Test Execution Monitor provides a simple framework
|
|
for program testing.<SPAN class="ref-to-top"><A href="#TOP"><IMG src="imgs/uarrow.gif" alt="reference to the top"></A></SPAN></P>
|
|
<H2><A name="Example">Example</A></H2>
|
|
<P class="1-line-indented">The example program shows six different ways to detect
|
|
and report an error in the add() function.</P>
|
|
<PRE class="code">#<SPAN class="reserv-word">include</SPAN> <<A href="../../../boost/test/test_tools.hpp">boost/test/test_tools.hpp</A>>
|
|
|
|
<SPAN class="cpp-type">int</SPAN> add( <SPAN class="cpp-type">int</SPAN> i, <SPAN class="cpp-type">int</SPAN> j ) { <SPAN class="reserv-word">return</SPAN> i+j; }
|
|
|
|
<SPAN class="cpp-type">int</SPAN> test_main( <SPAN class="cpp-type">int</SPAN>, <SPAN class="cpp-type">char </SPAN>*[] ) <SPAN class="comment">// note the name!</SPAN>
|
|
{
|
|
// six ways to detect and report the same error:
|
|
BOOST_CHECK( add( <SPAN class="literal">2</SPAN>,<SPAN class="literal">2</SPAN> ) == <SPAN class="literal">4</SPAN> ); <SPAN class="comment">// #1 continues on error</SPAN>
|
|
BOOST_REQUIRE( add( <SPAN class="literal">2</SPAN>,<SPAN class="literal">2</SPAN> ) == <SPAN class="literal">4</SPAN> ); <SPAN class="comment">// #2 throws on error</SPAN>
|
|
<SPAN class="reserv-word">if</SPAN>( add( <SPAN class="literal">2</SPAN>,<SPAN class="literal">2</SPAN> ) != <SPAN class="literal">4</SPAN> )
|
|
BOOST_ERROR( <SPAN class="literal">"Ouch..."</SPAN> ); <SPAN class="comment">// #3 continues on error</SPAN>
|
|
<SPAN class="reserv-word">if</SPAN>( add( <SPAN class="literal">2</SPAN>,<SPAN class="literal">2</SPAN> ) != <SPAN class="literal">4</SPAN> )
|
|
BOOST_FAIL( <SPAN class="literal">"Ouch..."</SPAN> ); <SPAN class="comment">// #4 throws on error</SPAN>
|
|
<SPAN class="reserv-word">if</SPAN>( add( <SPAN class="literal">2</SPAN>,<SPAN class="literal">2</SPAN> ) != <SPAN class="literal">4</SPAN> ) throw <SPAN class="literal">"Oops..."</SPAN>; <SPAN class="comment">// #5 throws on error</SPAN>
|
|
|
|
<SPAN class="reserv-word">return</SPAN> add( <SPAN class="literal">2</SPAN>, <SPAN class="literal">2</SPAN> ) == <SPAN class="literal">4</SPAN> ? <SPAN class="literal">0</SPAN> : <SPAN class="literal">1</SPAN>; <SPAN class="comment">// #6 returns error code</SPAN>
|
|
}</PRE>
|
|
<P><B>Approach #1</B> uses the BOOST_CHECK tool, which displays an error message
|
|
on std::cout that includes the expression that failed, the source file name, and
|
|
the source file line number. It also increments an error count. At program termination,
|
|
the error count will be displayed automatically by the Test Execution Monitor.</P>
|
|
<P><B>Approach #2</B> using the BOOST_REQUIRE tool, is similar to #1, except that
|
|
after displaying the error, an exception is thrown, to be caught by the Test Execution
|
|
Monitor. This approach is suitable when writing a explicit test program, and the
|
|
error would be so severe as to make further testing impractical. BOOST_REQUIRE
|
|
differs from the C++ Standard Library's assert() macro in that it is always generated,
|
|
and channels error detection into the uniform Test Execution Monitor reporting
|
|
procedure.</P>
|
|
<P><B>Approaches #3 and #4</B> are similar to #1 and #2 respectively, except that
|
|
the error detection is coded separately. This is most useful when the specific
|
|
condition being tested is not indicative of the reason for failure.</P>
|
|
<P><B>Approach #5</B> throws an exception, which will be caught and reported by the
|
|
Test Execution Monitor. This approach is suitable for both production and test
|
|
code, in libraries or not. The error message displayed when the exception is caught
|
|
will be most meaningful if the exception is derived from std::exception, or is
|
|
a char* or std::string.</P>
|
|
<P><B>Approach #6</B> uses a return value to inform the caller of the error. This
|
|
approach is particularly suitable for integrating existing test code with the test
|
|
tools library. Although it works fine with the Boost Program Execution Monitor
|
|
or Test Execution Monitor libraries, and is very useful for running existing code
|
|
under them, most C++ experts prefer using exceptions for error reporting.
|
|
<SPAN class="ref-to-top"><A href="#TOP"><IMG src="imgs/uarrow.gif" alt="reference to the top"></A></SPAN></P>
|
|
<H2>The Test <A name="MonitorCompilation">Execution Monitor compilation</A></H2>
|
|
<P class="1-line-indented">The Test Execution Monitor is supplied as an offline library
|
|
and should be compiled and linked with a test program. Following files, that are
|
|
located in the Boost Test Library src directory, compose the component:</P>
|
|
<P class="indented"> <A href="../src/execution_monitor.cpp">execution_monitor.cpp</A><BR>
|
|
<A href="../src/test_tools.cpp">test_tools.cpp</A><BR>
|
|
<A href="../src/unit_test_log.cpp">unit_test_log.cpp</A><BR>
|
|
<A href="../src/unit_test_parameters.cpp">unit_test_parameters.cpp</A><BR>
|
|
<A href="../src/unit_test_monitor.cpp">unit_test_monitor.cpp</A><BR>
|
|
<A href="../src/unit_test_result.cpp">unit_test_result.cpp</A><BR>
|
|
<A href="../src/unit_test_suite.cpp">unit_test_suite.cpp</A><BR>
|
|
<A href="../src/test_main.cpp">test_main.cpp</A></P>
|
|
<P class="1-line-indented">You also have a choice to include all files
|
|
constituting the Test Execution Monitor directly into your test module.
|
|
Use <a href="../../../boost/test/included/test_exec_monitor.hpp"><boost/test/included/test_exec_monitor.hpp></a>
|
|
for this porpose.</P>
|
|
<P></P>
|
|
<H2><A name="Tests">Example and Test</A> Programs</H2>
|
|
<P class="indented"> <A href="../example/test_exec_example.cpp">test_exec_example.cpp</A><BR>
|
|
<A href="../test/test_exec_fail1.cpp">test_exec_fail1.cpp</A><BR>
|
|
<A href="../test/test_exec_fail2.cpp">test_exec_fail2.cpp</A><BR>
|
|
<A href="../test/test_exec_fail3.cpp">test_exec_fail3.cpp</A><BR>
|
|
<A href="../test/test_exec_fail4.cpp">test_exec_fail4.cpp</A> </P>
|
|
<SPAN class="ref-to-top"><A href="#TOP"><IMG src="imgs/uarrow.gif" alt="reference to the top"></A></SPAN>
|
|
<H2><A name="Rationale">Rationale</A></H2>
|
|
|
|
<P class="1-line-indented">How should a test program report errors? Displaying an
|
|
error message is an obvious possibility:</P>
|
|
<PRE class="code"><SPAN class="reserv-word">if</SPAN>( something_bad_detected )
|
|
std::cout << "something bad has been detected" << std::endl;</PRE>
|
|
<P class="1-line-indented">But that requires inspection of the program's output after
|
|
each run to determine if an error occurred. Since test programs are often run as
|
|
part of a regression test suite, human inspection of output to detect error messages
|
|
is too time consuming and unreliable. Test frameworks like GNU/expect can do the
|
|
inspections automatically, but are overly complex for simple testing.</P>
|
|
<P class="1-line-indented">A better simple way to report errors is for the test program
|
|
to return EXIT_SUCCESS (normally 0) if the test program completes satisfactorily,
|
|
and EXIT_FAILURE if an error is detected. This allows a simple regression test
|
|
script to automatically and unambiguous detect success or failure. Further appropriate
|
|
actions such as creating an HTML table or emailing an alert can be taken by the
|
|
script, and can be modified as desired without having to change the actual C++
|
|
test programs.</P>
|
|
<P class="1-line-indented">A testing protocol based on a policy of test programs
|
|
returning EXIT_SUCCESS or EXIT_FAILURE does not require any supporting tools; the
|
|
C++ language and standard library are sufficient. The programmer must remember,
|
|
however, to catch all exceptions and convert them to program exits with non-zero
|
|
return codes. The programmer must also remember to not use the standard library
|
|
assert() macro for test code, because on some systems it results in undesirable
|
|
side effects like a message requiring manual intervention.</P>
|
|
<P class="1-line-indented">The Test Execution Monitor automates those tasks, yet
|
|
can be ignored by programmers who prefer to implement the zero return testing protocol
|
|
themselves.</P>
|
|
<H2><A name="Design">Design</A></H2>
|
|
<P>The <A href="test_lib_design.htm">Boost Test Library Design</A> document describes
|
|
the relationship between Boost Test Library components.
|
|
<SPAN class="ref-to-top"><A href="#TOP"><IMG src="imgs/uarrow.gif" alt="reference to the top"></A></SPAN></P>
|
|
<BR>
|
|
<DIV class="footer">
|
|
<P>© Beman Dawes 2000, <A href='mailto:rogeeff@emailaccount.com'>Gennadiy Rozental</A>
|
|
2001-2002 </P>
|
|
<P>Revised: 29 September, 2002</P>
|
|
</DIV>
|
|
</TD>
|
|
</TR>
|
|
</TABLE>
|
|
</DIV>
|
|
</BODY>
|
|
</HTML>
|
|
|