mirror of
https://github.com/boostorg/test.git
synced 2026-02-11 00:02:11 +00:00
418 lines
19 KiB
XML
418 lines
19 KiB
XML
<?xml version="1.0" encoding="utf-8"?>
|
|
<!DOCTYPE part PUBLIC "-//Boost//DTD BoostBook XML V1.0//EN" "../../../../tools/boostbook/dtd/boostbook.dtd" [
|
|
<!ENTITY utf "<acronym>UTF</acronym>">
|
|
]>
|
|
<section id="utf" last-revision="$Date$">
|
|
<title>Boost Test Library: The Unit Test Framework</title>
|
|
<titleabbrev>The Unit Test Framework</titleabbrev>
|
|
|
|
<section id="utf.intro">
|
|
<title>Introduction</title>
|
|
|
|
<epigraph>
|
|
<attribution>XP maxim</attribution>
|
|
<simpara>
|
|
The acceptance test makes the customer satisfied that the software provides the business value that makes them
|
|
willing to pay for it. The unit test makes the programmer satisfied that the software does what the programmer
|
|
thinks it does
|
|
</simpara>
|
|
</epigraph>
|
|
|
|
<para role="first-line-indented">
|
|
What is the first thing you need to do when you start working on new library/class/program? That's right -
|
|
you need to start with the unit test module (I hope you all gave this answer!). Occasional, simple test may be
|
|
implemented using asserts. But any professional developer soon finds this approach lacking. It becomes clear that
|
|
it's too time-consuming and tedious for simple, but repetitive unit testing tasks and it's too inflexible for
|
|
most nontrivial ones.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
<firstterm id="utf.def">The Boost Test Library Unit Test Framework</firstterm> (further in the documentation
|
|
referred by the acronym <acronym id="utf.def.ref">UTF</acronym>) provides both an easy to use and flexible solution
|
|
to this problem domain: C++ unit test implementation and organization.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
Unit testing tasks arise during many different stages of software development: from initial project implementation
|
|
to its maintenance and later revisions. These tasks differ in their complexity and purpose and accordingly are
|
|
approached differently by different developers. The wide spectrum of tasks in a problem domain cause many
|
|
requirements (sometimes conflicting) to be placed on a unit testing framework. These include:
|
|
</para>
|
|
|
|
<itemizedlist mark ="square">
|
|
<listitem>
|
|
<simpara>
|
|
Writing a unit test module should be simple and obvious for new users.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
The framework should allow advanced users to perform nontrivial tests.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
Test module should be able to have many small test cases and developer should be able to group them into test
|
|
suites.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
At the beginning of the development users want to see verbose and descriptive error message, whereas during the
|
|
regression testing they just want to know if any tests failed.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
For a small test modules run time should prevail over compilation time: user don't want to wait a minute to
|
|
compile a test that takes a second to run.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
For long and complex tests users want to be able to see the test progress.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
Simplest tests shouldn't require an external library.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
For long term usage users of a unit test framework should be able to build it as a standalone library.
|
|
</simpara>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para role="first-line-indented">
|
|
The &utf; design is based on above rationale and provides versatile facilities to:
|
|
</para>
|
|
|
|
<itemizedlist mark ="square">
|
|
<listitem>
|
|
<simpara>
|
|
Simplify writing test cases by using various <link linkend="utf.user-guide.testing-tools.reference">testing tools</link>.
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><link linkend="utf.user-guide.test-organization">Organize test cases</link> into a test tree.</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
Relieve you from messy error detection, reporting duties and framework runtime parameters processing.
|
|
</simpara>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para role="first-line-indented">
|
|
The &utf; keeps track of all passed/failed testing tools <link linkend="test-assertion.def">assertions</link>,
|
|
provides an ability to check the <link linkend="utf.user-guide.test-output.progress">test progress</link>
|
|
and generates a <link linkend="utf.user-guide.test-output.results-report">result report</link> in several different
|
|
formats. The &utf; supplies command line test runners that initialize the framework and run the requested tests.
|
|
Depending on the selected <link linkend="utf.compilation.flags">compilation flags</link> the function main()
|
|
default implementation, that invoke the supplied test runner, can be generated automatically as well.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
The &utf; is intended to be used both for a simple and non trivial testing. It is not intended to be used with
|
|
production code. In this case the <ulink url="under_construction.html">Program Execution Monitor</ulink> is more
|
|
suitable.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
Given the largely differing requirements of new and advanced users, it is clear that the &utf; must provide both
|
|
simple, easy-to-use interfaces with limited customization options and advanced interfaces, which allow unit testing
|
|
to be fully customized. Accordingly the material provided in this documentation is split into two sections:
|
|
</para>
|
|
|
|
<itemizedlist mark ="upper-roman">
|
|
<listitem>
|
|
<simpara>
|
|
<link linkend="utf.user-guide">The User's Guide</link>: covers all functionality that doesn't require
|
|
knowledge of the &utf; internals
|
|
</simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara>
|
|
<ulink url="under_construction.html">The Advanced User's Guide</ulink>: covers all implementation details
|
|
required for a user to understand the advanced customization options available in the &utf;, and for a user
|
|
interested in extending the testing framework.
|
|
</simpara>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para role="first-line-indented">
|
|
For those interested in getting started quickly please visit <link linkend="utf.examples">collection of
|
|
examples</link> presented in this documentation.
|
|
</para>
|
|
</section>
|
|
|
|
<xi:include href="utf.tutorials.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
|
|
|
|
<section id="utf.compilation">
|
|
<title>The &utf; compilation variants and procedures</title>
|
|
<titleabbrev>Compilation</titleabbrev>
|
|
|
|
<section id="utf.compilation.impl">
|
|
<title/>
|
|
|
|
<para role="first-line-indented">
|
|
The &utf; is comparatively complicated component and is implemented in close to hundred header and source files,
|
|
so for long term usage the preferable solution is to build the &utf; as a reusable standalone library.
|
|
Depending on your platform this may save you a significant time during test module compilation and doesn't
|
|
really require that much effort <footnote><simpara>If you are using Visual studio compilers do not forget to
|
|
set a subsystem to console when you build test modules. You can do it either in project properties or by setting
|
|
command line /SUBSYTEM:CONSOLE. Number of people reported link error caused specifically by this omission</simpara></footnote>.
|
|
<ulink url="http://boost.org/more/getting_started/index.html">Boost Getting started</ulink> tells you how to get
|
|
pre-built libraries for some platforms. If available, this is the easiest option and you can ignore standalone
|
|
library compilation instructions below.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
Following files constitute the &utf; implementation: <!--TO FIX: explain cpp/ipp -->
|
|
</para>
|
|
|
|
<itemizedlist html-class="files-list">
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/debug.ipp"><filename>debug.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/compiler_log_formatter.ipp"><filename>compiler_log_formatter.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/exception_safety.ipp"><filename>exception_safety.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/execution_monitor.ipp"><filename>execution_monitor.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/framework.ipp"><filename>framework.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/interaction_based.ipp"><filename>interaction_based.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/logged_expectations.ipp"><filename>logged_expectations.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/plain_report_formatter.ipp"><filename>plain_report_formatter.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/progress_monitor.ipp"><filename>progress_monitor.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/results_collector.ipp"><filename>results_collector.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/results_reporter.ipp"><filename>results_reporter.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/unit_test_log.ipp"><filename>unit_test_log.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/unit_test_main.ipp"><filename>unit_test_main.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/unit_test_monitor.ipp"><filename>unit_test_monitor.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/unit_test_parameters.ipp"><filename>unit_test_parameters.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/unit_test_suite.ipp"><filename>unit_test_suite.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/xml_log_formatter.ipp"><filename>xml_log_formatter.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
<listitem>
|
|
<simpara><ulink url="../../../../boost/test/impl/xml_report_formatter.ipp"><filename>xml_report_formatter.cpp</filename></ulink></simpara>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</section>
|
|
|
|
<section id="utf.compilation.procedure">
|
|
<title>Compilation procedures</title>
|
|
|
|
<para role="first-line-indented">
|
|
In comparison with many other boost libraries, which are completely implemented in header files, compilation and
|
|
linking with the &utf; may require additional steps. The &utf; presents you with options to either
|
|
<link linkend="utf.compilation.standalone">built and link with a standalone library</link> or
|
|
<link linkend="utf.compilation.direct-include">include the implementation directly</link> into a test module.
|
|
If you opt to use the library the &utf; headers implement the <link linkend="utf.compilation.auto-linking">
|
|
auto-linking support</link>. The compilation of the &utf; library and a test module can be configured using the
|
|
following compilation flags.
|
|
</para>
|
|
|
|
<table id="utf.compilation.flags">
|
|
<title>The &utf; compilation flags</title>
|
|
<tgroup cols="2">
|
|
<colspec colname="c1"/>
|
|
<colspec colname="c2"/>
|
|
<thead>
|
|
<row>
|
|
<entry>Flag</entry>
|
|
<entry>Usage</entry>
|
|
</row>
|
|
</thead>
|
|
<tbody>
|
|
<row>
|
|
<entry id="utf.flag.dyn-link">BOOST_TEST_DYN_LINK</entry>
|
|
|
|
<entry>Define this flag to build/use dynamic library.</entry>
|
|
</row>
|
|
<row>
|
|
<entry id="utf.flag.no-lib">BOOST_TEST_NO_LIB</entry>
|
|
|
|
<entry>Define this flag to prevent auto-linking.</entry>
|
|
</row>
|
|
<row>
|
|
<entry id="utf.flag.no-main">BOOST_TEST_NO_MAIN</entry>
|
|
|
|
<entry>Define this flag to prevent function main() implementation generation.</entry>
|
|
</row>
|
|
<row>
|
|
<entry id="utf.flag.main">BOOST_TEST_MAIN</entry>
|
|
|
|
<entry>
|
|
Define this flag to generate an empty test module initialization function and in case of
|
|
<link linkend="utf.user-guide.dynamic-lib-runner">dynamic library variant</link> default function main()
|
|
implementation as well.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry id="utf.flag.module">BOOST_TEST_MODULE</entry>
|
|
|
|
<entry>
|
|
Define this flag to generate the test module initialization function, which uses the defined value to name
|
|
the master test suite. In case of <link linkend="utf.user-guide.dynamic-lib-runner">dynamic library variant</link>
|
|
default function main() implementation is generated as well
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry id="utf.flag.alt-init-api">BOOST_TEST_ALTERNATIVE_INIT_API</entry>
|
|
|
|
<entry>
|
|
Define this flag to generate the switch to the alternative test module initialization API.
|
|
</entry>
|
|
</row>
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
<para role="first-line-indented">
|
|
Further in documentation you are going to see in details when and how these flags should be used.
|
|
</para>
|
|
</section>
|
|
|
|
<section id="utf.compilation.standalone">
|
|
<title>Standalone library compilation</title>
|
|
|
|
<para role="first-line-indented">
|
|
If you opted to link your program with the standalone library, you need to build it first. To build a standalone
|
|
library the all C++ files (.cpp), that constitute &utf; <link linkend="pem.impl">implementation</link> need to be
|
|
listed as source files in your makefile<footnote><simpara>There are varieties of make systems that can be used. To name
|
|
a few: <acronym>GNU</acronym> make (and other make clones) and build systems integrated into <acronym>IDE</acronym>s
|
|
(for example Microsoft Visual Studio). The Boost preferred solution is Boost.Build system that is based on top of
|
|
bjam tool. Make systems require some kind of configuration file that lists all files that constitute the library
|
|
and all build options. For example the makefile that is used by make, or the Microsoft Visual Studio project file,
|
|
Jamfile is used by Boost.Build. For the sake of simplicity let's call this file the makefile.</simpara></footnote>.
|
|
</para>
|
|
|
|
<para role="first-line-indented">
|
|
The Jamfile for use with Boost.Build system is supplied in <filename class="directory">libs/test/build</filename>
|
|
directory. The &utf; can be built as either <link id="prg-exec-monitor.compilation.standalone.static">static</link>
|
|
or <link id="prg-exec-monitor.compilation.standalone.dynamic">dynamic</link> library.
|
|
</para>
|
|
|
|
<section id="utf.compilation.standalone.static">
|
|
<title>Static library compilation</title>
|
|
|
|
<para role="first-line-indented">
|
|
No special build options or macro definitions are required to build the static library. Using the Boost.Build
|
|
system you can build the static library with the following command from
|
|
<filename class="directory">libs/test/build</filename> directory:
|
|
</para>
|
|
|
|
<cmdsynopsis>
|
|
<!-- TO FIX -->
|
|
<command>bjam</command>
|
|
<arg>-sTOOLS=<your-tool-name></arg>
|
|
<arg choice="req">-sBUILD=boost_unit_test_framework</arg>
|
|
</cmdsynopsis>
|
|
|
|
<para role="first-line-indented">
|
|
Also on Windows you can use the Microsoft Visual Studio .NET project file provided.
|
|
</para>
|
|
</section>
|
|
|
|
<section id="utf.compilation.standalone.dynamic">
|
|
<title>Dynamic library compilation</title>
|
|
|
|
<para role="first-line-indented">
|
|
To build the dynamic library<footnote><simpara>What is meant by the term dynamic library is a <firstterm>dynamically
|
|
loaded library</firstterm>, alternatively called a <firstterm>shared library</firstterm>.</simpara></footnote> you
|
|
need to add <xref linkend="utf.flag.dyn-link" endterm="utf.flag.dyn-link"/> to the list of macro definitions in the
|
|
makefile. Using the Boost.Build system you can build the dynamic library with the following command from
|
|
<filename class="directory">libs/test/build</filename> directory:
|
|
</para>
|
|
|
|
<cmdsynopsis>
|
|
<!-- TO FIX -->
|
|
<command>bjam</command>
|
|
<arg>-sTOOLS=<your-tool-name></arg>
|
|
<arg choice="req">-sBUILD=boost_unit_test_framework</arg>
|
|
</cmdsynopsis>
|
|
|
|
<para role="first-line-indented">
|
|
Also on Windows you can use the Microsoft Visual Studio .NET project file provided.
|
|
</para>
|
|
|
|
<important>
|
|
<simpara>
|
|
For test module to successfully link with the dynamic library the flag
|
|
<xref linkend="utf.flag.dyn-link" endterm="utf.flag.dyn-link"/> needs to be defined both during dynamic library build
|
|
and during test module compilation.
|
|
</simpara>
|
|
</important>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="utf.compilation.auto-linking">
|
|
<title>Support of the auto-linking feature</title>
|
|
<titleabbrev>Auto-linking support</titleabbrev>
|
|
|
|
<para role="first-line-indented">
|
|
For the Microsoft family of compilers the &utf; provides an ability to automatically select proper library name
|
|
and add it to the list of objects to be linked with. By default this feature is enabled. To disable it you have
|
|
to define the flag <xref linkend="utf.flag.no-lib" endterm="utf.flag.no-lib"/>. For more details on the auto-linking
|
|
feature implementation and configuration you should consult the
|
|
<ulink url="under_construction.html">appropriate documentation</ulink>.
|
|
</para>
|
|
</section>
|
|
|
|
<section id="utf.compilation.direct-include">
|
|
<title>Including the &utf; directly into your test module</title>
|
|
<titleabbrev>Direct include</titleabbrev>
|
|
|
|
<para role="first-line-indented">
|
|
If you prefer to avoid the standalone library compilation you can either include all files that constitute the
|
|
static library in your test module's makefile or include them as a part of a test module's source file.
|
|
To facilitate the later variant the &utf; presents the
|
|
<link linkend="utf.user-guide.single-header-variant">single-header usage variant</link>. In either case no special
|
|
build options or macro definitions are required to be added to your compilation options list by default. But the
|
|
same flags that can be used for the <link linkend="utf.compilation.standalone">standalone library compilation</link>
|
|
are applicable in this case. Though, obviously, neither <xref linkend="pem.flag.dyn-link" endterm="pem.flag.dyn-link"/>
|
|
nor <xref linkend="pem.flag.no-lib" endterm="pem.flag.no-lib"/> are applicable. This solution may not be the
|
|
best choice in a long run, since it requires the &utf; sources recompilation for every test module you use it with
|
|
and for every change of a test module you are working on. In a result your testing cycle time may increase. If it
|
|
become tiresome, I recommend switching to one of the prebuilt library usage variants.
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<xi:include href="utf.user-guide.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
|
|
<xi:include href="utf.usage-recommendations.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
|
|
<xi:include href="utf.examples.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
|
|
</section>
|