2
0
mirror of https://github.com/boostorg/test.git synced 2026-01-30 08:22:12 +00:00
Files
test/doc/src/utf.testing-tools.xml
Gennadiy Rozental 601e292135 some cleanup
[SVN r81190]
2012-11-05 05:56:05 +00:00

1169 lines
50 KiB
XML

<?xml version="1.0" encoding="utf-8" ?><!DOCTYPE chapter PUBLIC "-//Boost//DTD BoostBook XML V1.0//EN" "../../../../tools/boostbook/dtd/boostbook.dtd" [
<!ENTITY utf "<acronym>UTF</acronym>">
]>
<section id="utf.testing-tools">
<title>The &utf; testing tools &hellip; or tester's toolbox for all occasions</title>
<titleabbrev>Testing tools</titleabbrev>
<section id="utf.testing-tools.intro">
<title>Introduction</title>
<para role="first-line-indented">
The &utf;'s supplies a toolbox of testing tools to ease creation and maintenance of test programs and
provide a uniform error reporting mechanism. The toolbox supplied in most part in a form of macro and function
declarations. While the functions can be called directly, the usual way to use testing tools is via convenience
macros. All macros arguments are calculated once, so it's safe to pass complex expressions in their place.
All tools automatically supply an error location: a file name and a line number. The testing tools are intended
for unit test code rather than library or production code, where throwing exceptions, using assert(),
<classname>boost::concept_check</classname> or <macroname>BOOST_STATIC_ASSERT</macroname>() may be more suitable
ways to detect and report errors. For list of all supplied testing tools and usage examples see the reference.
</para>
</section>
<section id="utf.testing-tools.flavors">
<title>Testing tools flavors</title>
<para role="first-line-indented">
All the tools are supplied in three flavors(levels): <firstterm>WARN</firstterm>, <firstterm>CHECK</firstterm> and
<firstterm>REQUIRE</firstterm>. For example: <macroname>BOOST_WARN_EQUAL</macroname>,
<macroname>BOOST_CHECK_EQUAL</macroname>, <macroname>BOOST_REQUIRE_EQUAL</macroname>. If an assertion designated by
the tool passes, confirmation message can be printed in log output<footnote><simpara>to manage what messages appear
in the test log stream set the proper <link linkend="utf.user-guide.test-output.log">log
level</link></simpara></footnote>. If an assertion designated by the tool failed, depending on the level following
will happened<footnote><simpara>in some cases log message can be slightly different to reflect failed tool
specifics</simpara></footnote>:
</para>
<table id="utf.testing-tools.levels-diffs">
<title>Testing tools levels differences</title>
<tgroup cols="4">
<colspec colnum="1" colname="col1" />
<colspec colnum="2" colname="col2" />
<colspec colnum="3" colname="col3" />
<colspec colnum="4" colname="col4" />
<thead>
<row>
<entry>Level</entry>
<entry>Test log content</entry>
<entry>Errors counter</entry>
<entry>Test execution</entry>
</row>
</thead>
<tbody>
<row>
<entry>WARN</entry>
<entry>
warning in <replaceable>&lt;test case name&gt;</replaceable>: condition
<replaceable>&lt;assertion description&gt;</replaceable> is not satisfied
</entry>
<entry>not affected</entry>
<entry>continues</entry>
</row>
<row>
<entry>CHECK</entry>
<entry>
error in <replaceable>&lt;test case name&gt;</replaceable>: test
<replaceable>&lt;assertion description&gt;</replaceable> failed
</entry>
<entry>increased</entry>
<entry>continues</entry>
</row>
<row>
<entry>REQUIRE</entry>
<entry>
fatal error in <replaceable>&lt;test case name&gt;</replaceable>: critical test
<replaceable>&lt;assertion description&gt;</replaceable> failed
</entry>
<entry>increased</entry>
<entry>aborts</entry>
</row>
</tbody>
</tgroup>
</table>
<para role="first-line-indented">
Regularly you should use CHECK level tools to implement your assertions. You can use WARN level tools to validate
aspects less important then correctness: performance, portability, usability etc. You should use REQUIRE level
tools only if continuation of the test case doesn't make sense if this assertions fails.
</para>
</section>
<section id="utf.testing-tools.output-test">
<title>Output testing tool</title>
<para role="first-line-indented">
How do you perform correctness test for <code>operator&lt;&lt;( <classname>std::ostream</classname>&amp;, ... )</code>
operations? You can print into the standard output stream and manually check that it is matching your expectations.
Unfortunately, this is not really acceptable for the regression testing and doesn't serve a long term purpose of a
unit test. You can use <classname>std::stringstream</classname> and compare resulting output buffer with the
expected pattern string, but you are required to perform several additional operations with every check you do. So it
becomes tedious very fast. The class <firstterm><classname>output_test_stream</classname></firstterm> is designed to
automate these tasks for you. This is a simple, but powerful tool for testing standard
<classname>std::ostream</classname> based output operation. The class <classname>output_test_stream</classname>
complies to <classname>std::ostream</classname> interface so it can be used in place of any
<classname>std::ostream</classname> parameter. It provides several test methods to validate output content,
including test for match to expected output content or test for expected output length. Flushing, synchronizing,
string comparison and error message generation is automated by the tool implementation.
</para>
<para role="first-line-indented">
All <classname>output_test_stream</classname> validation methods by default flush the stream once check is performed.
If you want to perform several checks with the same output, specify parameter <firstterm>flush_stream</firstterm>
with value false. This parameter is supported on all comparison methods.
</para>
<para role="first-line-indented">
In some cases manual generation of expected output is either too time consuming or is impossible at all bacause
of sheer volume. What we need in cases like this is to be able to check once manually that the output is as expected
and to be able in a future check that it stays the same. To help manage this logic the class
<classname>output_test_stream</classname> allows matching output content versus specified pattern file and produce
pattern file based on successful test run.
</para>
<para role="first-line-indented">
Detailed specification of class <classname>output_test_stream</classname> is covered in reference section.
</para>
<section id="utf.testing-tools.output-test.usage">
<title>Usage</title>
<para role="first-line-indented">
There are two ways to employ the class <classname>output_test_stream</classname>: explicit output checks and
pattern file matching.
</para>
</section>
<btl-example name="example28">
<title>output_test_stream usage with explicit output checks</title>
<para role="first-line-indented">
Use the instance of class output_test_stream as an output stream and check output content using tool's methods.
Note use of <literal>false</literal> to prevent output flushing in first two invocation of check functions. Unless
you want to perform several different checks for the same output you wouldn't need to use it though. Your
test will look like a serious of output operators followed by one check. And so on again. Try to perform checks as
frequently as possible. It not only simplifies patterns you compare with, but also allows you to more closely
identify possible source of failure.
</para>
</btl-example>
<btl-example name="example29">
<title>output_test_stream usage for pattern file matching</title>
<para role="first-line-indented">
Even simpler: no need to generate expected patterns. Though you need to keep the pattern file all the time somewhere
around. Your testing will look like a serious of output operators followed by match pattern checks repeated several
times. Try to perform checks as frequently as possible, because it allows you to more closely identify possible source
of failure. Content of the pattern file is:
</para>
<simpara>
<literallayout>i=2
File: test.cpp Line: 14</literallayout>
</simpara>
</btl-example>
</section>
<section id="utf.testing-tools.custom-predicate">
<title>Custom predicate support</title>
<para role="first-line-indented">
Even though supplied testing tools cover wide range of possible checks and provide detailed report on cause of error
in some cases you may want to implement and use custom predicate that perform complex check and produce intelligent
report on failure. To satisfy this need testing tools implement custom predicate support. There two layers of custom
predicate support implemented by testing tools toolbox: with and without custom error message generation.
</para>
<para role="first-line-indented">
The first layer is supported by BOOST_CHECK_PREDICATE family of testing tools. You can use it to check any custom
predicate that reports the result as boolean value. The values of the predicate arguments are reported by the tool
automatically in case of failure.
</para>
<btl-example name="example30">
<title>Custom predicate support using BOOST_CHECK_PREDICATE</title>
</btl-example>
<para role="first-line-indented">
To use second layer your predicate have to return
<classname>boost::test_tools::predicate_result</classname>. This class encapsulates boolean result value along
with any error or information message you opt to report.
</para>
<para role="first-line-indented">
Usually you construct the instance of class <classname>boost::test_tools::predicate_result</classname> inside your
predicate function and return it by value. The constructor expects one argument - the boolean result value. The
constructor is implicit, so you can simply return boolean value from your predicate and
<classname>boost::test_tools::predicate_result</classname> is constructed automatically to hold your value and empty
message. You can also assign boolean value to the constructed instance. You can check the current predicate value by
using <methodname>operator!</methodname>() or directly accessing public read-only property p_predicate_value. The
error message is stored in public read-write property p_message.
</para>
<btl-example name="example31">
<title>Custom predicate support using class predicate_result</title>
</btl-example>
</section>
<section id="utf.testing-tools.fpv-comparison">
<title>Floating-point comparison algorithms</title>
<para role="first-line-indented">
In most cases it is unreasonable to use an <code>operator==(...)</code> for a floating-point values equality check.
The simple, absolute value comparison based, solution for a floating-point values <varname>u</varname>,
<varname>v</varname> and a tolerance &egr;:
</para>
<btl-equation index="1">
|<varname>u</varname> &minus; <varname>v</varname>| &le; &egr;
</btl-equation>
<simpara>
does not produce expected results in many circumstances - specifically for very small or very big values (See
<xref linkend="bbl.Squassabia"/> for examples). The &utf; implements floating-point comparison algorithm that is
based on the more confident solution first presented in <xref linkend="bbl.KnuthII"/>:
</simpara>
<btl-equation index="2">
|<varname>u</varname> &minus; <varname>v</varname>| &le; &egr; &times; |<varname>u</varname>| &and; |<varname>u</varname> &minus; <varname>v</varname>| &le; &egr; &times; |<varname>v</varname>|
</btl-equation>
<simpara>
defines a <firstterm>very close with tolerance &egr;</firstterm> relationship between <varname>u</varname> and <varname>v</varname>
</simpara>
<btl-equation index="3">
|<varname>u</varname> &minus; <varname>v</varname>| &le; &egr; &times; |<varname>u</varname>| &or; |<varname>u</varname> &minus; <varname>v</varname>| &le; &egr; &times; |<varname>v</varname>|
</btl-equation>
<simpara>
defines a <firstterm>close enough with tolerance &egr;</firstterm> relationship between <varname>u</varname> and <varname>v</varname>
</simpara>
<para role="first-line-indented">
Both relationships are commutative but are not transitive. The relationship defined by inequations
(<ulink linkend="utf.testing-tools.fpv-comparison.eq.2">2</ulink>) is stronger
that the relationship defined by inequations (<ulink linkend="utf.testing-tools.fpv-comparison.eq.3">3</ulink>)
(i.e. (<ulink linkend="utf.testing-tools.fpv-comparison.eq.2">2</ulink>) &rArr;
(<ulink linkend="utf.testing-tools.fpv-comparison.eq.3">3</ulink>)). Because of the multiplication in the right side
of inequations, that can cause an unwanted underflow condition, the implementation is using modified version of the
inequations (<ulink linkend="utf.testing-tools.fpv-comparison.eq.2">2</ulink>) and
(<ulink linkend="utf.testing-tools.fpv-comparison.eq.3">3</ulink>) where all underflow, overflow conditions can be
guarded safely:
</para>
<btl-equation index="4">
|<varname>u</varname> &minus; <varname>v</varname>| &sol; |<varname>u</varname>| &le; &egr; &and; |<varname>u</varname> &minus; <varname>v</varname>| / |<varname>v</varname>| &le; &egr;
</btl-equation>
<btl-equation index="5">
|<varname>u</varname> &minus; <varname>v</varname>| &sol; |<varname>u</varname>| &le; &egr; &or; |<varname>u</varname> &minus; <varname>v</varname>| / |<varname>v</varname>| &le; &egr;
</btl-equation>
<para role="first-line-indented">
Checks based on equations (<ulink linkend="utf.testing-tools.fpv-comparison.eq.4">4</ulink>) and
(<ulink linkend="utf.testing-tools.fpv-comparison.eq.5">5</ulink>) are implemented by two predicates with
alternative interfaces: binary predicate <classname>close_at_tolerance</classname><footnote><simpara>check type
and tolerance value are fixed at predicate construction time</simpara></footnote> and predicate with four arguments
<classname>check_is_close</classname><footnote><simpara>check type and tolerance value are the arguments of the
predicate</simpara></footnote>.
</para>
<para role="first-line-indented">
While equations (<ulink linkend="utf.testing-tools.fpv-comparison.eq.4">4</ulink>) and
(<ulink linkend="utf.testing-tools.fpv-comparison.eq.5">5</ulink>) in general are preferred for the general floating
point comparison check over equation (<ulink linkend="utf.testing-tools.fpv-comparison.eq.1">1</ulink>), they are
unusable for the test on closeness to zero. The later check is still might be useful in some cases and the &utf;
implements an algorithm based on equation (<ulink linkend="utf.testing-tools.fpv-comparison.eq.1">1</ulink>) in
binary predicate <classname>check_is_small</classname><footnote><simpara><varname>v</varname> is zero</simpara></footnote>.
</para>
<para role="first-line-indented">
On top of the generic, flexible predicates the &utf; implements macro based family of tools
<macroname>BOOST_CHECK_CLOSE</macroname> and <macroname>BOOST_CHECK_SMALL</macroname>. These tools limit the check
flexibility to strong-only checks, but automate failed check arguments reporting.
</para>
<section id="utf.testing-tools.fpv-comparison.tolerance-selection">
<title>Tolerance selection considerations</title>
<para role="first-line-indented">
In case of absence of domain specific requirements the value of tolerance can be chosen as a sum of the predicted
upper limits for "relative rounding errors" of compared values. The "rounding" is the operation by which a real
value 'x' is represented in a floating-point format with 'p' binary digits (bits) as the floating-point value 'X'.
The "relative rounding error" is the difference between the real and the floating point values in relation to real
value: |x-X|/|x|. The discrepancy between real and floating point value may be caused by several reasons:
</para>
<itemizedlist>
<listitem><simpara>Type promotion</simpara></listitem>
<listitem><simpara>Arithmetic operations</simpara></listitem>
<listitem><simpara>Conversion from a decimal presentation to a binary presentation</simpara></listitem>
<listitem><simpara>Non-arithmetic operation</simpara></listitem>
</itemizedlist>
<para role="first-line-indented">
The first two operations proved to have a relative rounding error that does not exceed &frac12; &times;
"machine epsilon value" for the appropriate floating point type (represented by
<classname>std::numeric_limits</classname>&lt;FPT&gt;::epsilon()). Conversion to binary presentation, sadly, does
not have such requirement. So we can't assume that float 1.1 is close to real 1.1 with tolerance &frac12;
&times; "machine epsilon value" for float (though for 11./10 we can). Non arithmetic operations either do not have a
predicted upper limit relative rounding errors. Note that both arithmetic and non-arithmetic operations might also
produce others "non-rounding" errors, such as underflow/overflow, division-by-zero or 'operation errors'.
</para>
<para role="first-line-indented">
All theorems about the upper limit of a rounding error, including that of &frac12; &times; epsilon, refer only to
the 'rounding' operation, nothing more. This means that the 'operation error', that is, the error incurred by the
operation itself, besides rounding, isn't considered. In order for numerical software to be able to actually
predict error bounds, the IEEE754 standard requires arithmetic operations to be 'correctly or exactly rounded'.
That is, it is required that the internal computation of a given operation be such that the floating point result
is the exact result rounded to the number of working bits. In other words, it is required that the computation used
by the operation itself doesn't introduce any additional errors. The IEEE754 standard does not require same behavior
from most non-arithmetic operation. The underflow/overflow and division-by-zero errors may cause rounding errors
with unpredictable upper limits.
</para>
<para role="first-line-indented">
At last be aware that &frac12; &times; epsilon rules are not transitive. In other words combination of two
arithmetic operations may produce rounding error that significantly exceeds 2 &times; &frac12; &times; epsilon. All
in all there are no generic rules on how to select the tolerance and users need to apply common sense and domain/
problem specific knowledge to decide on tolerance value.
</para>
<para role="first-line-indented">
To simplify things in most usage cases latest version of algorithm below opted to use percentage values for
tolerance specification (instead of fractions of related values). In other words now you use it to check that
difference between two values does not exceed x percent.
</para>
<para role="first-line-indented">
For more reading about floating-point comparison see references below.
</para>
</section>
<bibliography id="bbl.fpv-comparison">
<title>A floating-point comparison related references</title>
<bibliodiv id="bbl.fpv-comparison.books"><title>Books</title>
<biblioentry id="bbl.KnuthII">
<abbrev>KnuthII</abbrev>
<title>The art of computer programming (vol II)</title>
<author><firstname>Donald. E.</firstname><surname>Knuth</surname></author>
<copyright><year>1998</year><holder>Addison-Wesley Longman, Inc.</holder></copyright>
<isbn>0-201-89684-2</isbn>
<publisher><publishername>Addison-Wesley Professional; 3 edition</publishername></publisher>
</biblioentry>
<biblioentry id="bbl.Kulisch">
<abbrev>Kulisch</abbrev>
<biblioset relation="section">
<title>Rounding near zero</title>
</biblioset>
<biblioset relation="book">
<title><ulink url="http://www.amazon.com/Advanced-Arithmetic-Digital-Computer-Kulisch/dp/3211838708">Advanced Arithmetic for the Digital Computer</ulink></title>
<author><firstname>Ulrich W</firstname><surname>Kulisch</surname></author>
<copyright><year>2002</year><holder>Springer, Inc.</holder></copyright>
<isbn>0-201-89684-2</isbn>
<publisher><publishername>Springer; 1 edition</publishername></publisher>
</biblioset>
</biblioentry>
</bibliodiv>
<bibliodiv id="bbl.fpv-comparison.periodicals"><title>Periodicals</title>
<biblioentry id="bbl.Squassabia">
<abbrev>Squassabia</abbrev>
<title><ulink url="http://www.adtmag.com/joop/carticle.aspx?ID=396">Comparing Floats: How To Determine if Floating Quantities Are Close Enough Once a Tolerance Has Been Reached</ulink></title>
<author><firstname>Alberto</firstname><surname>Squassabia</surname></author>
<biblioset relation="journal">
<title>C++ Report</title>
<issuenum>March 2000</issuenum>.
</biblioset>
</biblioentry>
<biblioentry id="bbl.Becker">
<abbrev>Becker</abbrev>
<biblioset relation="article">
<title>The Journeyman's Shop: Trap Handlers, Sticky Bits, and Floating-Point Comparisons</title>
<author><firstname>Pete</firstname><surname>Becker</surname></author>
</biblioset>
<biblioset relation="journal">
<title>C/C++ Users Journal</title>
<issuenum>December 2000</issuenum>.
</biblioset>
</biblioentry>
</bibliodiv>
<bibliodiv id="bbl.fpv-comparison.publications"><title>Publications</title>
<biblioentry id="bbl.Goldberg">
<abbrev>Goldberg</abbrev>
<biblioset relation="article">
<title><ulink url="http://citeseer.ist.psu.edu/goldberg91what.html">What Every Computer Scientist Should Know About Floating-Point Arithmetic</ulink></title>
<author><firstname>David</firstname><surname>Goldberg</surname></author>
<copyright><year>1991</year><holder>Association for Computing Machinery, Inc.</holder></copyright>
<pagenums>150-230</pagenums>
</biblioset>
<biblioset relation="journal">
<title>Computing Surveys</title>
<issuenum>March</issuenum>.
</biblioset>
</biblioentry>
<biblioentry id="bbl.Langlois">
<abbrev>Langlois</abbrev>
<title><ulink url="http://www.inria.fr/rrrt/rr-3967.html">From Rounding Error Estimation to Automatic Correction with Automatic Differentiation</ulink></title>
<author><firstname>Philippe</firstname><surname>Langlois</surname></author>
<copyright><year>2000</year></copyright>
<issn>0249-6399</issn>
</biblioentry>
<biblioentry id="bbl.Kahan">
<abbrev>Kahan</abbrev>
<title><ulink url="http://www.cs.berkeley.edu/~wkahan/">Lots of information on William Kahan home page</ulink></title>
<author><firstname>William</firstname><surname>Kahan</surname></author>
</biblioentry>
</bibliodiv>
</bibliography>
</section>
<section id="utf.testing-tools.reference">
<title>The &utf; testing tools reference</title>
<titleabbrev>Reference</titleabbrev>
<inline-reference id="utf.testing-tools.reference.body">
<refentry name="BOOST_&lt;level&gt;">
<inline-synopsis>
<macro name="BOOST_WARN" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
</macro>
<macro name="BOOST_CHECK" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
</macro>
<macro name="BOOST_REQUIRE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate
value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The
expression gets evaluated only once, so it's safe to pass complex expression for validation.
</para>
<btl-example name="example34">
<title>BOOST_&lt;level&gt; usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_MESSAGE</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_BITWISE_EQUAL">
<inline-synopsis>
<macro name="BOOST_WARN_BITWISE_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_BITWISE_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_BITWISE_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to perform bitwise comparison of two values. The check shows all positions where left and
right value's bits mismatch.
</para>
<para role="first-line-indented">
The first parameter is the left compared value. The second parameter is the right compared value. Parameters are
not required to be of the same type, but warning is issued if their type's size does not coincide.
</para>
<btl-example name="example33">
<title>BOOST_&lt;level&gt;_BITWISE_EQUAL usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_EQUAL</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_CLOSE">
<inline-synopsis>
<macro name="BOOST_WARN_CLOSE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_CHECK_CLOSE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_REQUIRE_CLOSE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to check on closeness using strong relationship defined by the predicate
<functionname>check_is_close</functionname>( left, right, tolerance ). To check for the weak relationship use
<ref>BOOST_&lt;level&gt;_PREDICATE</ref> family of tools with explicit <functionname>check_is_close</functionname>
invocation.
</para>
<para>
The first parameter is the <emphasis>left</emphasis> compared value. The second parameter is the
<emphasis>right</emphasis> compared value. Last third parameter defines the tolerance for the comparison in
<link linkend="utf.testing-tools.fpv-comparison.tolerance-selection"><emphasis role="bold">percentage units</emphasis></link>.
</para>
<note>
<simpara>
It is required for left and right parameters to be of the same floating point type. You will need to explicitly
resolve any type mismatch to select which type to use for comparison.
</simpara>
</note>
<note>
<simpara>
Note that to use these tools you need to include additional header floating_point_comparison.hpp.
</simpara>
</note>
<btl-example name="example42">
<title>BOOST_&lt;level&gt;_CLOSE usage with very small values</title>
</btl-example>
<btl-example name="example43">
<title>BOOST_&lt;level&gt;_CLOSE usage with very big values</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_CLOSE_FRACTION</ref>, <ref>BOOST_&lt;level&gt;_SMALL</ref>, <ref>BOOST_&lt;level&gt;_EQUAL</ref>,
<link linkend="utf.testing-tools.fpv-comparison">Floating point comparison algorithms</link>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_CLOSE_FRACTION">
<inline-synopsis>
<macro name="BOOST_WARN_CLOSE_FRACTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_CHECK_CLOSE_FRACTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_REQUIRE_CLOSE_FRACTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
<macro-parameter name="tolerance"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to check on closeness using strong relationship defined by the predicate
<functionname>check_is_close</functionname>( left, right, tolerance ). To check for the weak relationship use
<ref>BOOST_&lt;level&gt;_PREDICATE</ref> family of tools with explicit <functionname>check_is_close</functionname>
invocation.
</para>
<para>
The first parameter is the <emphasis>left</emphasis> compared value. The second parameter is the
<emphasis>right</emphasis> compared value. Last third parameter defines the tolerance for the comparison as
<link linkend="utf.testing-tools.fpv-comparison.tolerance-selection"><emphasis role="bold">fraction of absolute
values being compared</emphasis></link>.
</para>
<note>
<simpara>
It is required for left and right parameters to be of the same floating point type. You will need to explicitly
resolve any type mismatch to select which type to use for comparison.
</simpara>
</note>
<note>
<simpara>
Note that to use these tools you need to include additional header floating_point_comparison.hpp.
</simpara>
</note>
<btl-example name="example44">
<title>BOOST_&lt;level&gt;_CLOSE_FRACTION usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_CLOSE</ref>, <ref>BOOST_&lt;level&gt;_SMALL</ref>, <ref>BOOST_&lt;level&gt;_EQUAL</ref>,
<link linkend="utf.testing-tools.fpv-comparison">Floating point comparison algorithms</link>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_EQUAL">
<inline-synopsis>
<macro name="BOOST_WARN_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_EQUAL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left == right ).
The difference is that the mismatched values are reported as well.
</para>
<note>
<simpara>
It is bad idea to use these tools to compare floating point values. Use <ref>BOOST_&lt;level&gt;_CLOSE</ref> or
<ref>BOOST_&lt;level&gt;_CLOSE_FRACTION</ref> tools instead.
</simpara>
</note>
<btl-example name="example35">
<title>BOOST_&lt;level&gt;_EQUAL usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>, <ref>BOOST_&lt;level&gt;_CLOSE</ref>, <ref>BOOST_&lt;level&gt;_NE</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_EQUAL_COLLECTIONS">
<inline-synopsis>
<macro name="BOOST_WARN_EQUAL_COLLECTIONS" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left_begin"/>
<macro-parameter name="left_end"/>
<macro-parameter name="right_begin"/>
<macro-parameter name="right_end"/>
</macro>
<macro name="BOOST_CHECK_EQUAL_COLLECTIONS" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left_begin"/>
<macro-parameter name="left_end"/>
<macro-parameter name="right_begin"/>
<macro-parameter name="right_end"/>
</macro>
<macro name="BOOST_REQUIRE_EQUAL_COLLECTIONS" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left_begin"/>
<macro-parameter name="left_end"/>
<macro-parameter name="right_begin"/>
<macro-parameter name="right_end"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to perform an element by element comparison of two collections. They print all mismatched
positions, collection elements at these positions and check that the collections have the same size. The first two
parameters designate begin and end of the first collection. The two parameters designate begin and end of the
second collection.
</para>
<btl-example name="example36">
<title>BOOST_&lt;level&gt;_EQUAL_COLLECTIONS usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_EQUAL</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_EXCEPTION">
<inline-synopsis>
<macro name="BOOST_WARN_EXCEPTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
<macro-parameter name="predicate"/>
</macro>
<macro name="BOOST_CHECK_EXCEPTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
<macro-parameter name="predicate"/>
</macro>
<macro name="BOOST_REQUIRE_EXCEPTION" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
<macro-parameter name="predicate"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to perform an exception detection and validation check. Tools execute the supplied expression
and validate that it throws an exception of supplied class (or the one derived from it) that complies with the
supplied predicate. If the expression throws any other unrelated exception, doesn't throw at all or
predicate evaluates to false, check fails. In comparison with <ref>BOOST_&lt;level&gt;_THROW</ref> tools these
allow performing more fine-grained checks. For example: make sure that an expected exception has specific
error message.
</para>
<btl-example name="example37">
<title>BOOST_&lt;level&gt;_EXCEPTION usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_THROW</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_GE">
<inline-synopsis>
<macro name="BOOST_WARN_GE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_GE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_GE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left &gt;= right ).
The difference is that the argument values are reported as well.
</para>
<btl-example name="example57">
<title>BOOST_&lt;level&gt;_GE usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_LE</ref>, <ref>BOOST_&lt;level&gt;_LT</ref>, <ref>BOOST_&lt;level&gt;_GT</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_GT">
<inline-synopsis>
<macro name="BOOST_WARN_GT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_GT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_GT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left &gt; right ).
The difference is that the argument values are reported as well.
</para>
<btl-example name="example58">
<title>BOOST_&lt;level&gt;_GT usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_LE</ref>, <ref>BOOST_&lt;level&gt;_LT</ref>, <ref>BOOST_&lt;level&gt;_GE</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_LE">
<inline-synopsis>
<macro name="BOOST_WARN_LE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_LE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_LE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left &lt;= right ).
The difference is that the argument values are reported as well.
</para>
<btl-example name="example55">
<title>BOOST_&lt;level&gt;_LE usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_LT</ref>, <ref>BOOST_&lt;level&gt;_GE</ref>, <ref>BOOST_&lt;level&gt;_GT</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_LT">
<inline-synopsis>
<macro name="BOOST_WARN_LT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_LT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_LT" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left &lt; right ).
The difference is that the argument values are reported as well.
</para>
<btl-example name="example56">
<title>BOOST_&lt;level&gt;_LT usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_LE</ref>, <ref>BOOST_&lt;level&gt;_GE</ref>, <ref>BOOST_&lt;level&gt;_GT</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_MESSAGE">
<inline-synopsis>
<macro name="BOOST_WARN_MESSAGE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="message"/>
</macro>
<macro name="BOOST_CHECK_MESSAGE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="message"/>
</macro>
<macro name="BOOST_REQUIRE_MESSAGE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="message"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools perform exactly the same check as <ref>BOOST_&lt;level&gt;</ref> tools. The only difference is that
instead of generating an error/confirm message these use the supplied one.
</para>
<para>
The first parameter is the boolean expression. The second parameter is the message reported in case of check
failure. The message argument can be constructed of components of any type supporting the
<code>std::ostream&amp; operator&lt;&lt;(std::ostream&amp;)</code>.
</para>
<btl-example name="example38">
<title>BOOST_&lt;level&gt;_MESSAGE usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_NE">
<inline-synopsis>
<macro name="BOOST_WARN_NE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_CHECK_NE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
<macro name="BOOST_REQUIRE_NE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="left"/>
<macro-parameter name="right"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Check performed by these tools is the same as the one performed by <ref>BOOST_&lt;level&gt;</ref>( left != right ).
The difference is that the matched values are reported as well.
</para>
<btl-example name="example54">
<title>BOOST_&lt;level&gt;_NE usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_EQUAL</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_NO_THROW">
<inline-synopsis>
<macro name="BOOST_WARN_NO_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
</macro>
<macro name="BOOST_CHECK_NO_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
</macro>
<macro name="BOOST_REQUIRE_NO_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to perform a "no throw" check. Tools execute the supplied expression and validate that it does
not throw any exceptions. Error would be reported by the framework even if the statement appear directly in test
case body and throw any exception. But these tools allow proceeding further with test case in case of failure.
</para>
<para>
If check is successful, tools may produce a confirmation message, in other case they produce an error message in
a form "error in &lt;test case name&gt;exception was thrown by &lt;expression&gt;.
</para>
<para>
The only parameter is an expression to execute. You can use do-while(0) block if you want to execute more than one
statement.
</para>
<btl-example name="example39">
<title>BOOST_&lt;level&gt;_NO_THROW usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_THROW</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_PREDICATE">
<inline-synopsis>
<macro name="BOOST_WARN_PREDICATE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="arguments_list"/>
</macro>
<macro name="BOOST_CHECK_PREDICATE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="arguments_list"/>
</macro>
<macro name="BOOST_REQUIRE_PREDICATE" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="predicate"/>
<macro-parameter name="arguments_list"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on
predicate arity defined by the configurable macro <macroname>BOOST_TEST_MAX_PREDICATE_ARITY</macroname>). To
validate zero arity predicate use <ref>BOOST_&lt;level&gt;</ref> tools. In other cases prefer theses tools. The
advantage of these tools is that they show arguments values in case of predicate failure.
</para>
<para>
The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped
in round brackets (BOOST_PP sequence format).
</para>
<btl-example name="example40">
<simpara>
Note difference in error log from
</simpara>
<title>BOOST_&lt;level&gt;_PREDICATE usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_SMALL">
<inline-synopsis>
<macro name="BOOST_WARN_SMALL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="value"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_CHECK_SMALL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="value"/>
<macro-parameter name="tolerance"/>
</macro>
<macro name="BOOST_REQUIRE_SMALL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="value"/>
<macro-parameter name="tolerance"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value
of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness
it's preferable to use <ref>BOOST_&lt;level&gt;_CLOSE</ref> tools instead.
</para>
<para role="first-line-indented">
The first parameter is the value to check. The second parameter is the tolerance.
</para>
<note>
<simpara>
Note that to use these tools you need to include additional header floating_point_comparison.hpp.
</simpara>
</note>
<btl-example name="example41">
<title>BOOST_&lt;level&gt;_SMALL usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;_CLOSE</ref>, <ref>BOOST_&lt;level&gt;_CLOSE_FRACTION</ref>,
<link linkend="utf.testing-tools.fpv-comparison">Floating point comparison algorithms</link>
</seealso>
</refentry>
<refentry name="BOOST_&lt;level&gt;_THROW">
<inline-synopsis>
<macro name="BOOST_WARN_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
</macro>
<macro name="BOOST_CHECK_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
</macro>
<macro name="BOOST_REQUIRE_THROW" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="expression"/>
<macro-parameter name="exception"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
These tools are used to perform an exception detection check. Tools execute the supplied expression and validate
that it throws an exception of supplied class (or the one derived from it) or it's child. If the statement
throws any other unrelated exception or doesn't throw at all, check fails.
</para>
<para>
If check is successful, the tool produces a confirmation message, in other case it produces an error message in a
form "error in <replaceable>test case name</replaceable>: exception <replaceable>exception</replaceable> expected.
</para>
<para role="first-line-indented">
The first parameter is the expression to execute. Use do-while(0) block if you want to execute more than one
statement. The second parameter is an expected exception.
</para>
<btl-example name="example45">
<title>BOOST_&lt;level&gt;_THROW usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;NO_THROW</ref>
</seealso>
</refentry>
<refentry name="BOOST_ERROR">
<inline-synopsis>
<macro name="BOOST_ERROR" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="message"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
BOOST_ERROR tool behave the same way as <code>BOOST_CHECK_MESSAGE( false, message )</code>. This tool is used for
an unconditional error counter increasing and message logging.
</para>
<para>
The only tool's parameter is an error message to log.
</para>
<btl-example name="example46">
<title>BOOST_ERROR usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>
</seealso>
</refentry>
<refentry name="BOOST_FAIL">
<inline-synopsis>
<macro name="BOOST_FAIL" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="message"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
BOOST_FAIL behave the same way as <code>BOOST_REQUIRE_MESSAGE( false, message )</code>. This tool is used for an
unconditional error counter increasing, message logging and the current test case aborting.
</para>
<para>
The only tool's parameter is an error message to log.
</para>
<btl-example name="example47">
<title>BOOST_FAIL usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>
</seealso>
</refentry>
<refentry name="BOOST_IS_DEFINED">
<inline-synopsis>
<macro name="BOOST_IS_DEFINED" kind="functionlike" ref-id="utf.testing-tools.reference">
<macro-parameter name="symbol"/>
</macro>
</inline-synopsis>
<para role="first-line-indented">
Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose
is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with
<ref>BOOST_&lt;level&gt;</ref> to perform and log validation. Macros of any arity could be checked. To check the
macro definition with non-zero arity specify dummy arguments for it. See below for example.
</para>
<para>
The only tool's parameter is a preprocessor symbol that gets validated.
</para>
<btl-example name="example48">
<title>BOOST_IS_DEFINED usage</title>
</btl-example>
<seealso>
<ref>BOOST_&lt;level&gt;</ref>
</seealso>
</refentry>
</inline-reference>
</section>
</section>