mirror of
https://github.com/boostorg/website.git
synced 2026-01-27 19:32:16 +00:00
164 lines
6.2 KiB
HTML
164 lines
6.2 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
|
|
<head>
|
|
<title>Boost Test Policies and Protocols</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />
|
|
<link rel="icon" href="/favicon.ico" type="image/ico" />
|
|
<link rel="stylesheet" type="text/css" href=
|
|
"/style/section-development.css" />
|
|
<!--[if IE]> <style type="text/css"> body { behavior: url(/style/csshover.htc); } </style> <![endif]-->
|
|
</head>
|
|
|
|
<body>
|
|
<div id="heading">
|
|
<!--#include virtual="/common/heading.html" -->
|
|
</div>
|
|
|
|
<div id="body">
|
|
<div id="body-inner">
|
|
<div id="content">
|
|
<div class="section" id="intro">
|
|
<div class="section-0">
|
|
<div class="section-title">
|
|
<h1>Boost Test Policies and Protocols</h1>
|
|
</div>
|
|
|
|
<div class="section-body">
|
|
<p>The Boost libraries are intended to be both reliable and
|
|
portable. Every experienced programmer knows that means each
|
|
library must be tested against a suitable number of test cases,
|
|
on a wide range of platforms, and then tested again (regression
|
|
tested) every time a change is made and before every
|
|
release.</p>
|
|
|
|
<p>"Quality assurance based on a wide range of targeted tests"
|
|
as one of the key answers to C.A.R Hoare's question "How did
|
|
software get so reliable without proof."</p>
|
|
|
|
<h2>Regression test</h2>
|
|
|
|
<p>Boost uses an automatic <a href="regression.html">regression
|
|
test suite</a> which generates HTML <a href=
|
|
"../status/compiler_status.html">compiler status
|
|
tables</a>.</p>
|
|
|
|
<h2>Test Policy</h2>
|
|
|
|
<h3>Required</h3>
|
|
|
|
<ul>
|
|
<li>Every Boost library should supply one or more suitable
|
|
test programs to be exercised by the Boost <a href=
|
|
"regression.html">regression test suite</a>. In addition to
|
|
the usual compile-link-run tests expecting successful
|
|
completion, compile-only or compile-and-link-only tests may
|
|
be performed, and success for the test may be defined as
|
|
failure of the steps.</li>
|
|
|
|
<li>Test program execution must report errors by returning a
|
|
non-zero value. They may also write to stdout or stderr, but
|
|
that output should be relatively brief. Regardless of other
|
|
output, a non-zero return value is the only way the
|
|
regression test framework will recognize an error has
|
|
occurred. Note that test programs to be included in the
|
|
status tables must compile, link, and run quickly since the
|
|
tests are executed many, many, times.</li>
|
|
|
|
<li>Libraries with time consuming tests should be divided
|
|
into a fast-execution basic test program for the status
|
|
tables, and a separate full-coverage test program for
|
|
exhaustive test cases. The basic test should concentrate on
|
|
compilation issues so that the status tables accurately
|
|
reflect the library's likelihood of correct compilation on a
|
|
platform.</li>
|
|
|
|
<li>If for any reason the usual test policies do not apply to
|
|
a particular library, an alternate test strategy must be
|
|
implemented.</li>
|
|
|
|
<li>A <a href="regression.html#Adding_new_test">Jamfile</a>
|
|
to drive the regression tests for the library.</li>
|
|
</ul>
|
|
|
|
<h3>Optional (but highly recommended)</h3>
|
|
|
|
<p>The <a href="../libs/test/index.html">Boost Test Library</a>
|
|
provides many useful components which ease the construction of
|
|
test programs.</p>
|
|
|
|
<ul>
|
|
<li>Use the library's <a href=
|
|
"../libs/test/doc/components/test_tools/index.html">Test
|
|
Tools</a> for the construction of simple test programs that
|
|
do not need much structure.</li>
|
|
|
|
<li>Use the library's <a href=
|
|
"../libs/test/doc/components/utf/index.html">Unit Test
|
|
Framework</a> for the construction of more complex test
|
|
programs that need to be structured into individual tests and
|
|
test suites.</li>
|
|
</ul>
|
|
|
|
<h2>Suggested Protocol for Fixing Bugs or Adding Features.</h2>
|
|
|
|
<ul>
|
|
<li>First, add regression test cases that detects the bug or
|
|
tests the feature. Sometimes adding one case suggests similar
|
|
untested cases, and they are added too.</li>
|
|
|
|
<li>Second, for bugs, run the regression test and verify that
|
|
the bug is now detected.</li>
|
|
|
|
<li>Third, then, and only then, fix the bug or add the
|
|
feature.</li>
|
|
|
|
<li>Finally, rerun the full regression tests - sometimes the
|
|
change breaks something else.</li>
|
|
</ul>
|
|
|
|
<h2>History</h2>
|
|
|
|
<p><a href="regression.html#History">See Regression Test
|
|
History</a>.</p>
|
|
|
|
<h2>Acknowledgements</h2>
|
|
|
|
<p>Written by Beman Dawes. Jens Maurer, Paul Moore, Gary Powell
|
|
and Jeremy Siek contributed helpful suggestions.</p>
|
|
<hr />
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div id="sidebar">
|
|
<!--#include virtual="/common/sidebar-common.html" -->
|
|
<!--#include virtual="/common/sidebar-development.html" -->
|
|
</div>
|
|
|
|
<div class="clear"></div>
|
|
</div>
|
|
</div>
|
|
|
|
<div id="footer">
|
|
<div id="footer-left">
|
|
<div id="revised">
|
|
<p>Revised $Date$</p>
|
|
</div>
|
|
|
|
<div id="copyright">
|
|
<p>Copyright Beman Dawes 2001.</p>
|
|
</div><!--#include virtual="/common/footer-license.html" -->
|
|
</div>
|
|
|
|
<div id="footer-right">
|
|
<!--#include virtual="/common/footer-banners.html" -->
|
|
</div>
|
|
|
|
<div class="clear"></div>
|
|
</div>
|
|
</body>
|
|
</html>
|