mirror of
https://github.com/boostorg/website.git
synced 2026-01-19 04:42:17 +00:00
171 lines
6.6 KiB
HTML
171 lines
6.6 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
|
|
<head>
|
|
<title>Boost Test Policies and Protocols</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />
|
|
<link rel="icon" href="/favicon.ico" type="image/ico" />
|
|
<link rel="stylesheet" type="text/css" href=
|
|
"../style-v2/section-development.css" />
|
|
<!--[if IE 7]> <style type="text/css"> body { behavior: url(/style-v2/csshover3.htc); } </style> <![endif]-->
|
|
</head><!--
|
|
Note: Editing website content is documented at:
|
|
https://www.boost.org/development/website_updating.html
|
|
-->
|
|
|
|
<body>
|
|
<div id="heading">
|
|
<!--#include virtual="/common/heading.html" -->
|
|
</div>
|
|
|
|
<div id="body">
|
|
<div id="body-inner">
|
|
<div id="content">
|
|
<div class="section" id="intro">
|
|
<div class="section-0">
|
|
<div class="section-title">
|
|
<h1>Boost Test Policies and Protocols</h1>
|
|
</div>
|
|
|
|
<div class="section-body">
|
|
<p>The Boost libraries are intended to be both reliable and
|
|
portable. Every experienced programmer knows that means each
|
|
library must be tested against a suitable number of test cases,
|
|
on a wide range of platforms, and then tested again (regression
|
|
tested) every time a change is made and before every
|
|
release.</p>
|
|
|
|
<p>"Quality assurance based on a wide range of targeted tests"
|
|
as one of the key answers to C.A.R Hoare's question "How did
|
|
software get so reliable without proof."</p>
|
|
|
|
<h2>Regression test</h2>
|
|
|
|
<p>Boost uses an automatic <a href=
|
|
"/doc/tools/regression/index.html">regression
|
|
test suite</a> which generates HTML <a href=
|
|
"/development/testing.html#RegressionTesting">compiler status
|
|
tables</a>.</p>
|
|
|
|
<h2>Test Policy</h2>
|
|
|
|
<h3>Required</h3>
|
|
|
|
<ul>
|
|
<li>Every Boost library should supply one or more suitable
|
|
test programs to be exercised by the Boost <a href=
|
|
"/doc/tools/regression/index.html">regression
|
|
test suite</a>. In addition to the usual compile-link-run
|
|
tests expecting successful completion, compile-only or
|
|
compile-and-link-only tests may be performed, and success for
|
|
the test may be defined as failure of the steps.</li>
|
|
|
|
<li>Test program execution must report errors by returning a
|
|
non-zero value. They may also write to stdout or stderr, but
|
|
that output should be relatively brief. Regardless of other
|
|
output, a non-zero return value is the only way the
|
|
regression test framework will recognize an error has
|
|
occurred. Note that test programs to be included in the
|
|
status tables must compile, link, and run quickly since the
|
|
tests are executed many, many, times.</li>
|
|
|
|
<li>Libraries with time consuming tests should be divided
|
|
into a fast-execution basic test program for the status
|
|
tables, and a separate full-coverage test program for
|
|
exhaustive test cases. The basic test should concentrate on
|
|
compilation issues so that the status tables accurately
|
|
reflect the library's likelihood of correct compilation on a
|
|
platform.</li>
|
|
|
|
<li>If for any reason the usual test policies do not apply to
|
|
a particular library, an alternate test strategy must be
|
|
implemented.</li>
|
|
|
|
<li>A <a href=
|
|
"/doc/tools/regression/index.html#Adding_new_test">
|
|
Jamfile</a> to drive the regression tests for the
|
|
library.</li>
|
|
</ul>
|
|
|
|
<h3>Optional (but highly recommended)</h3>
|
|
|
|
<p>The <a href="/doc/libs/release/libs/test/index.html">Boost
|
|
Test Library</a> provides many useful components which ease the
|
|
construction of test programs.</p>
|
|
|
|
<ul>
|
|
<li>Use the library's <a href=
|
|
"/doc/libs/release/libs/test/doc/html/boost_test/testing_tools.html">
|
|
Test Tools</a> for the construction of simple test programs
|
|
that do not need much structure.</li>
|
|
|
|
<li>Use the library's <a href=
|
|
"/doc/libs/release/libs/test/doc/html/boost_test/tests_organization.html">Unit
|
|
Test Framework</a> for the construction of more complex test
|
|
programs that need to be structured into individual tests and
|
|
test suites.</li>
|
|
</ul>
|
|
|
|
<h2>Suggested Protocol for Fixing Bugs or Adding Features.</h2>
|
|
|
|
<ul>
|
|
<li>First, add regression test cases that detects the bug or
|
|
tests the feature. Sometimes adding one case suggests similar
|
|
untested cases, and they are added too.</li>
|
|
|
|
<li>Second, for bugs, run the regression test and verify that
|
|
the bug is now detected.</li>
|
|
|
|
<li>Third, then, and only then, fix the bug or add the
|
|
feature.</li>
|
|
|
|
<li>Finally, rerun the full regression tests - sometimes the
|
|
change breaks something else.</li>
|
|
</ul>
|
|
|
|
<h2>History</h2>
|
|
|
|
<p><a href=
|
|
"/doc/tools/regression/index.html#History">See
|
|
Regression Test History</a>.</p>
|
|
|
|
<h2>Acknowledgements</h2>
|
|
|
|
<p>Written by Beman Dawes. Jens Maurer, Paul Moore, Gary Powell
|
|
and Jeremy Siek contributed helpful suggestions.</p>
|
|
<hr />
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div id="sidebar">
|
|
<!--#include virtual="/common/sidebar-common.html" -->
|
|
<!--#include virtual="/common/sidebar-development.html" -->
|
|
</div>
|
|
|
|
<div class="clear"></div>
|
|
</div>
|
|
</div>
|
|
|
|
<div id="footer">
|
|
<div id="footer-left">
|
|
<div id="revised">
|
|
<p>Revised $Date$</p>
|
|
</div>
|
|
|
|
<div id="copyright">
|
|
<p>Copyright Beman Dawes 2001.</p>
|
|
</div><!--#include virtual="/common/footer-license.html" -->
|
|
</div>
|
|
|
|
<div id="footer-right">
|
|
<!--#include virtual="/common/footer-banners.html" -->
|
|
</div>
|
|
|
|
<div class="clear"></div>
|
|
</div>
|
|
</body>
|
|
</html>
|