mirror of
https://github.com/boostorg/test.git
synced 2026-01-25 18:52:15 +00:00
171 lines
9.8 KiB
HTML
171 lines
9.8 KiB
HTML
<html>
|
|
|
|
<head>
|
|
<title>Test Execution Monitor</title>
|
|
<script language="javascript">var viso_path="js-lib"</script>
|
|
<script language="javascript" src="js-lib/core.js" > </script>
|
|
|
|
<script language="JavaScript">
|
|
JS.include( "btl.js" );
|
|
</script>
|
|
|
|
<script>put_screen_style();</script>
|
|
<link rel="stylesheet" type="text/css" href="style/btl_print.css" media="print" />
|
|
|
|
<meta http-equiv="Content-Language" content="en-us" />
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
|
|
</head>
|
|
|
|
<body onload="btl_menu_init()">
|
|
<div align="center">
|
|
<table class="body-table" cellspacing="3" >
|
|
<tr>
|
|
<td id="body">
|
|
|
|
<script language="Javascript">btl_header()</script>
|
|
|
|
<h1>Boost Test Library: Test Execution Monitor</h1>
|
|
|
|
<p class="page-toc">
|
|
<a href="#Introduction">Introduction</a><br>
|
|
<a href="#Benefits">Benefits</a><br>
|
|
<a href="#Introduction">Example</a><br>
|
|
<a href="#MonitorCompilation">Compilation</a><br>
|
|
<a href="#Tests">Test/Example Programs</a><br>
|
|
<a href="#Rationale">Rationale</a><br>
|
|
<a href="#Design">Design</a><br><br>
|
|
|
|
Also see: <a href="unit_test_framework.htm">Unit Test Framework</a>
|
|
</p>
|
|
|
|
<h2><a name="Introduction">Introduction</a></h2>
|
|
<p class="1-line-indented">The Boost Test Library's Test Execution Monitor provides a <b>main()</b> function which calls a user-supplied <b>test_main()</b>
|
|
function. The library supplied <b>main()</b> relieves users from messy error detection and reporting duties.
|
|
The Test Execution Monitor is intended for fairly simple test programs or to dig a problem in
|
|
an existent production code. <a href="prg_exec_monitor.htm">Program Execution Monitor</a> may be more suitable to monitor production (non-test) programs. <a href="unit_test_framework.htm">Unit Test Framework</a> may be more suitable for complex test programs. </p>
|
|
|
|
<h2><a name="Benefits">Benefits</a></h2>
|
|
<p class="1-line-indented">The Test Execution Monitor provides a simple framework for program testing.<script language="Javascript">put_ref_to_top()</script></p>
|
|
|
|
<h2><a name="Example">Example</a></h2>
|
|
<p class="1-line-indented">The example program shows six different ways to detect and report an error in the
|
|
<nobr>add()</nobr> function.</p>
|
|
<pre class="code">#<span class="reserv-word">include</span> <<a href="../../../boost/test/test_tools.hpp">boost/test/test_tools.hpp</a>>
|
|
|
|
<span class="cpp-type">int</span> add( <span class="cpp-type">int</span> i, <span class="cpp-type">int</span> j ) { <span class="reserv-word">return</span> i+j; }
|
|
|
|
<span class="cpp-type">int</span> test_main( <span class="cpp-type">int</span>, <span class="cpp-type">char </span>*[] ) <span class="comment">// note the name!</span>
|
|
{
|
|
// six ways to detect and report the same error:
|
|
BOOST_CHECK( add( <span class="literal">2</span>,<span class="literal">2</span> ) == <span class="literal">4</span> ); <span class="comment">// #1 continues on error</span>
|
|
BOOST_REQUIRE( add( <span class="literal">2</span>,<span class="literal">2</span> ) == <span class="literal">4</span> ); <span class="comment">// #2 throws on error</span>
|
|
<span class="reserv-word">if</span>( add( <span class="literal">2</span>,<span class="literal">2</span> ) != <span class="literal">4</span> )
|
|
BOOST_ERROR( <span class="literal">"Ouch..."</span> ); <span class="comment">// #3 continues on error</span>
|
|
<span class="reserv-word">if</span>( add( <span class="literal">2</span>,<span class="literal">2</span> ) != <span class="literal">4</span> )
|
|
BOOST_FAIL( <span class="literal">"Ouch..."</span> ); <span class="comment">// #4 throws on error</span>
|
|
<span class="reserv-word">if</span>( add( <span class="literal">2</span>,<span class="literal">2</span> ) != <span class="literal">4</span> ) throw <span class="literal">"Oops..."</span>; <span class="comment">// #5 throws on error</span>
|
|
|
|
<span class="reserv-word">return</span> add( <span class="literal">2</span>, <span class="literal">2</span> ) == <span class="literal">4</span> ? <span class="literal">0</span> : <span class="literal">1</span>; <span class="comment">// #6 returns error code</span>
|
|
}</pre>
|
|
|
|
<p><b>Approach #1</b> uses the BOOST_CHECK tool, which displays an error message
|
|
on std::cout that includes the expression that failed, the source file name, and
|
|
the source file line number. It also increments an error count. At program termination,
|
|
the error count will be displayed automatically by the Test Execution Monitor.</p>
|
|
<p><b>Approach #2</b> using the BOOST_REQUIRE tool, is similar to #1, except that after
|
|
displaying the error, an exception is thrown, to be caught by the Test Execution
|
|
Monitor. This approach is suitable when writing a
|
|
explicit test program, and the error would be so severe as to make further
|
|
testing impractical. BOOST_REQUIRE differs from the C++ Standard Library's
|
|
assert() macro in that it is always generated, and channels error detection into
|
|
the uniform Test Execution Monitor reporting procedure.</p>
|
|
<p><b>Approaches #3 and #4</b> are similar to #1 and #2 respectively, except
|
|
that the error detection is coded separately. This is most useful when the
|
|
specific condition being tested is not indicative of the reason for failure.</p>
|
|
<p><b>Approach #5</b> throws an exception, which will be caught and reported by the Test
|
|
Execution Monitor. This approach is suitable for both production and test code, in
|
|
libraries or not. The error message displayed when the exception is caught
|
|
will be most meaningful if the exception is derived from std::exception, or is a
|
|
char* or std::string.</p>
|
|
<p><b>Approach #6</b> uses a return value to inform the caller of the error. This approach is particularly suitable for
|
|
integrating existing test code with the test tools library. Although it
|
|
works fine with the Boost Program Execution Monitor or Test Execution Monitor libraries, and is very
|
|
useful for running existing code under them, most C++ experts prefer using exceptions for error reporting.
|
|
<script language="Javascript">put_ref_to_top()</script></p>
|
|
|
|
<h2>The Test <a name="MonitorCompilation">Execution Monitor compilation</a></h2>
|
|
<p class="1-line-indented">The Test Execution Monitor is supplied as an offline library and should be compiled and
|
|
linked with a test program. Following files, that are located in the Boost Test Library
|
|
src directory, compose the component:</p>
|
|
|
|
<p class="indented">
|
|
<a href="../src/execution_monitor.cpp">execution_monitor.cpp</a><br>
|
|
<a href="../src/test_tools.cpp">test_tools.cpp</a><br>
|
|
<a href="../src/unit_test_log.cpp">unit_test_log.cpp</a><br>
|
|
<a href="../src/unit_test_parameters.cpp">unit_test_parameters.cpp</a><br>
|
|
<a href="../src/unit_test_monitor.cpp">unit_test_monitor.cpp</a><br>
|
|
<a href="../src/unit_test_result.cpp">unit_test_result.cpp</a><br>
|
|
<a href="../src/unit_test_suite.cpp">unit_test_suite.cpp</a><br>
|
|
<a href="../src/test_main.cpp">test_main.cpp</a></p>
|
|
</p>
|
|
|
|
<h2><a name="Tests">Example and Test</a> Programs</h2>
|
|
<p class="indented">
|
|
<a href="../example/test_exec_example.cpp">test_exec_example.cpp</a><br>
|
|
<a href="../test/test_exec_fail1.cpp">test_exec_fail1.cpp</a><br>
|
|
<a href="../test/test_exec_fail2.cpp">test_exec_fail2.cpp</a><br>
|
|
<a href="../test/test_exec_fail3.cpp">test_exec_fail3.cpp</a><br>
|
|
<a href="../test/test_exec_fail4.cpp">test_exec_fail4.cpp</a>
|
|
</p>
|
|
|
|
<h2><a name="Rationale">Rationale</a><script language="Javascript">put_ref_to_top()</script></h2>
|
|
<p class="1-line-indented">How should a test program report errors? Displaying an error message is an obvious possibility:</p>
|
|
|
|
<pre class="code"><span class="reserv-word">if</span>( something_bad_detected )
|
|
std::cout << "something bad has been detected" << std::endl;</pre>
|
|
|
|
<p class="1-line-indented">But that requires inspection of the program's output after each run to
|
|
determine if an error occurred. Since test programs are often run as part
|
|
of a regression test suite, human inspection of output to detect error messages
|
|
is too time consuming and unreliable. Test frameworks like GNU/expect can do the
|
|
inspections automatically, but are overly complex for simple testing.</p>
|
|
<p class="1-line-indented">A better simple way to report errors is for the test program to return
|
|
EXIT_SUCCESS (normally 0) if the test program completes satisfactorily, and
|
|
EXIT_FAILURE if an error is detected. This allows a simple regression test script to
|
|
automatically and unambiguous detect success or failure. Further
|
|
appropriate actions such as creating an HTML table or emailing an alert can be
|
|
taken by the script, and can be modified as desired without having to change the
|
|
actual C++ test programs.</p>
|
|
<p class="1-line-indented">A testing protocol based on a policy of test programs returning EXIT_SUCCESS or
|
|
EXIT_FAILURE does not require any supporting tools; the C++ language and standard
|
|
library are sufficient. The programmer must remember, however, to catch
|
|
all exceptions and convert them to program exits with non-zero return codes. The
|
|
programmer must also remember to not use the standard library assert() macro for
|
|
test code, because on some systems it results in undesirable side effects like a message
|
|
requiring manual intervention.</p>
|
|
<p class="1-line-indented">The Test Execution Monitor automates those tasks, yet can be ignored by
|
|
programmers who prefer to implement the zero return testing protocol themselves.</p>
|
|
|
|
<h2><a name="Design">Design</a></h2>
|
|
<p>The <a href="test_lib_design.htm">Boost Test Library Design</a> document describes the relationship
|
|
between Boost Test Library components.
|
|
<script language="Javascript">put_ref_to_top()</script></p>
|
|
<hr>
|
|
<p>© Beman Dawes 2000,
|
|
<script language="Javascript">contact_addess()</script>
|
|
<script language="Javascript">get_copyright_date()</script>
|
|
</p>
|
|
|
|
<p>Revised
|
|
<!--webbot bot="Timestamp" S-Type="EDITED"
|
|
S-Format="%d %b %Y" startspan -->29 Jul 2002<!--webbot bot="Timestamp" endspan i-checksum="14998" -->
|
|
</p>
|
|
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
</body>
|
|
</html>
|
|
|