boost/test
2
0
Fork 0
mirror of https://github.com/boostorg/test.git synced 2026-01-27 07:22:11 +00:00
Code Issues Projects Releases Wiki Activity

Merge branch 'develop'

Browse Source 
* develop:
  Passing the pattern file as argument
  Warnings
  Change logs
  Documentation update for the new macros
  Fixing logging issues for JUnit with global fixtures
  Fixture setup/teardown now applied to BOOST_FIXTURE_TEST_CASE familly of macros
  Fixture setup/teardown detection for C++11 capable compilers
  Improving global initialization and fixtures
This commit is contained in:
Raffi Enficiaud
2017-06-26 16:31:07 +02:00
parent bf6d703643 0ea2e11944
commit 3d13638c41
51 changed files with 7463 additions and 312 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
build/CMakeLists.txt
View File

@@ -90,6 +90,7 @@ set(BOOST_UTF_SRC
${BOOST_TEST_ROOT_DIR}/src/progress_monitor.cpp
${BOOST_TEST_ROOT_DIR}/src/results_collector.cpp
${BOOST_TEST_ROOT_DIR}/src/results_reporter.cpp
${BOOST_TEST_ROOT_DIR}/src/test_framework_init_observer.cpp
${BOOST_TEST_ROOT_DIR}/src/test_tools.cpp
${BOOST_TEST_ROOT_DIR}/src/test_tree.cpp
${BOOST_TEST_ROOT_DIR}/src/unit_test_log.cpp

2
build/Jamfile.v2
View File

@@ -47,6 +47,7 @@ TEST_EXEC_MON_SOURCES =
progress_monitor
results_collector
results_reporter
test_framework_init_observer
test_main
test_tools
test_tree
@@ -69,6 +70,7 @@ UTF_SOURCES =
progress_monitor
results_collector
results_reporter
test_framework_init_observer
test_tools
test_tree
unit_test_log

24
doc/adv_scenarios/link_reference.qbk
View File

@@ -8,6 +8,7 @@
[section:link_references Build scenarios and behaviors]
[/-----------------------------------------------------------------]
[section:link_boost_test_main_macro `BOOST_TEST_MAIN`]
When defined, this macro creates a stub for the test module initialization (the main entry part). This
@@ -25,6 +26,7 @@ macro also expands properly into a `main` function in case the shared library va
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_module_macro `BOOST_TEST_MODULE`]
Serves the same purpose as the macro __BOOST_TEST_MAIN__ but, in addition, defines the name of the master test suite.
@@ -39,21 +41,24 @@ An example may be found [link ref_BOOST_TEST_MODULE here].
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_alternative_init_macro `BOOST_TEST_ALTERNATIVE_INIT_API`]
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_no_lib `BOOST_TEST_NO_LIB`]
Define this flag to prevent auto-linking.
[note The same flag is used for the __UTF__ and the __PEM__ components.]
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_dyn_link `BOOST_TEST_DYN_LINK`]
Define this flag to link against the __UTF__ shared library.
[note The same flag is used for the __UTF__ and the __PEM__ components.]
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_no_main `BOOST_TEST_NO_MAIN`]
Prevents the auto generation of the test module initialization functions. This macro is particularly relevant for
manually registered tests in conjunction with dynamic variant of the __UTF__. When defined, a `main` function
@@ -84,6 +89,23 @@ int main(int argc, char* argv[])
``
[endsect]
[/-----------------------------------------------------------------]
[section:link_boost_test_global_configuration `BOOST_TEST_GLOBAL_CONFIGURATION`]
Declares a class that will be constructed during the initialization of the test framework, and destructed afterwards.
The framework will not call any other member function than the constructor and destructor.
In particular the constructor and destructor will be called prior and after to the [link boost_test.tests_organization.fixtures.global global fixtures]
setup and teardown.
This facility is provided to perform additional configuration, in particular programmatic configuration
of the loggers and reporters. See [link boost_test.test_output.logging_api this section] for more details.
[warning No logging or any other call to the framework assertion is allowed in the constructor and destructor, as its purpose is
to set-up the loggers/reporters, and the assertions are calling the logging/reporting facility.
Any such assertion during the execution of the will result in the abortion of the test module .]
[endsect]
[/-----------------------------------------------------------------]
[section:config_disable_alt_stack `BOOST_TEST_DISABLE_ALT_STACK`]
Disables the support of the alternative stack.

15
doc/closing_chapters/change_log.qbk
View File

@@ -9,11 +9,22 @@
[h4 Boost.Test v3.6 / boost 1.65]
[h5 Breaking changes]
* __BOOST_GLOBAL_FIXTURE__ is flagged as deprecated and will be removed in a later version
* Using test assertions and macros is not allowed when used inside __BOOST_GLOBAL_FIXTURE__. Please use __BOOST_TEST_GLOBAL_FIXTURE__
instead (see below).
[h5 New features]
* VS2017 / C++17 compatibility (thanks to Daniela Engert)
* Deprecating __BOOST_GLOBAL_FIXTURE__ in favor of __BOOST_TEST_GLOBAL_FIXTURE__ and __BOOST_TEST_GLOBAL_CONFIGURATION__. This
helps separating the logic of the fixtures associated to the master test suite, from the one used for setting up the logging
and reporting facility, and results in a general cleaner design.
[h5 Bugfixes and feature requests]
# [pull_request 106] replace deprecated binders and adapters, and `random_shuffle` by more modern equivalents
# [ticket 5282] Test fixtures do not support virtual inheritance
# [ticket 5563] using a test macro in a global fixture crashes Boost.Test
# [ticket 11962] `BOOST_TEST_MESSAGE` in fixture constructor - invalid XML
[h4 Boost.Test v3.5 / boost 1.64]
@@ -24,10 +35,10 @@
[link boost_test.test_output.log_formats.test_log_output log-level] set between `success` and
`non-fatal error`, and defaults to `general information`.
* [link boost_test.test_output.log_formats.log_junit_format JUnit output format] is now more
performant in case a lot of checks are done in a test module.
performant in case a lot of checks are done in a test module.
[h5 Bugfixes and feature requests]
# [pull_request 107] `BOOST_NO_EXCEPTIONS` typo making `throw_exception` unusable under some circumstances
# [pull_request 107] `BOOST_NO_EXCEPTIONS` typo making `throw_exception` unusable under some circumstances
# [pull_request 108] Change capital variable names to lowercase
# [ticket 11756] boost.Test: non standards compliant use of `FE_*` macros (unable to compile boost test library on FPU-less arches) (reopened)
# [ticket 12540] Provide customisation point for printing types in tests

2
doc/examples/example20.run.cpp
View File

@@ -19,7 +19,7 @@ struct MyConfig {
//____________________________________________________________________________//
BOOST_GLOBAL_FIXTURE( MyConfig );
BOOST_TEST_GLOBAL_FIXTURE( MyConfig );
BOOST_AUTO_TEST_CASE( test_case )
{

4
doc/examples/example50.output
View File

@@ -4,5 +4,5 @@
*** 1 failures is detected in test suite "example"
> cat example.log
Running 1 test case...
test.cpp(20): error in "test_case": check false failed
//]
test.cpp(26): error in "test_case": check false failed
//]

20
doc/examples/example50.run-fail.cpp
View File

@@ -10,25 +10,19 @@
#include <boost/test/included/unit_test.hpp>
#include <fstream>
struct MyConfig
{
MyConfig() : test_log( "example.log" )
{
boost::unit_test::unit_test_log.set_stream( test_log );
struct MyConfig {
MyConfig() : test_log( "example.log" ) {
boost::unit_test::unit_test_log.set_stream( test_log );
}
~MyConfig()
{
boost::unit_test::unit_test_log.set_stream( std::cout );
~MyConfig() {
boost::unit_test::unit_test_log.set_stream( std::cout );
}
std::ofstream test_log;
};
BOOST_GLOBAL_FIXTURE( MyConfig );
BOOST_TEST_GLOBAL_CONFIGURATION( MyConfig );
BOOST_AUTO_TEST_CASE( test_case )
{
BOOST_AUTO_TEST_CASE( test_case ) {
BOOST_TEST( false );
}
//]

4
doc/examples/example52.output
View File

@@ -1,4 +1,4 @@
//[example_output
> example --report_level=no
<TestLog><Error file="test.cpp" line="18">check false failed</Error></TestLog>
//]
<TestLog><Error file="test.cpp" line="23">check false failed</Error></TestLog>
//]

8
doc/examples/example52.run-fail.cpp
View File

@@ -11,17 +11,15 @@
using namespace boost::unit_test;
struct MyConfig {
MyConfig()
{
MyConfig() {
unit_test_log.set_format( OF_XML );
}
~MyConfig() {}
};
BOOST_GLOBAL_FIXTURE( MyConfig );
BOOST_TEST_GLOBAL_CONFIGURATION( MyConfig );
BOOST_AUTO_TEST_CASE( test_case0 )
{
BOOST_AUTO_TEST_CASE( test_case0 ) {
BOOST_TEST( false );
}
//]

13
doc/examples/fixture_04.output Normal file
View File

@@ -0,0 +1,13 @@
//[example_output
> fixture_04 --log_level=message
Running 2 test cases...
ctor fixture i=0
setup fixture i=0
running test_case1
running test_case2
./fixture_04.run-fail.cpp:42: error: in "test_case2": check MyGlobalFixture::i == 3 has failed [1 != 3]
teardown fixture i=1
dtor fixture i=3
*** 1 failure is detected in the test module "fixture_04"
//]

44
doc/examples/fixture_04.run-fail.cpp Normal file
View File

@@ -0,0 +1,44 @@
// (C) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
//[example_code
#define BOOST_TEST_MODULE fixture_04
#include <boost/test/included/unit_test.hpp>
struct MyGlobalFixture {
MyGlobalFixture() {
BOOST_TEST_MESSAGE( "ctor fixture i=" << i );
}
void setup() {
BOOST_TEST_MESSAGE( "setup fixture i=" << i );
i++;
}
void teardown() {
BOOST_TEST_MESSAGE( "teardown fixture i=" << i );
i += 2;
}
~MyGlobalFixture() {
BOOST_TEST_MESSAGE( "dtor fixture i=" << i );
}
static int i;
};
int MyGlobalFixture::i = 0;
BOOST_TEST_GLOBAL_FIXTURE( MyGlobalFixture );
BOOST_AUTO_TEST_CASE(test_case1)
{
BOOST_TEST_MESSAGE("running test_case1");
BOOST_TEST(MyGlobalFixture::i == 1);
}
BOOST_AUTO_TEST_CASE(test_case2)
{
BOOST_TEST_MESSAGE("running test_case2");
BOOST_TEST(MyGlobalFixture::i == 3);
}
//]

4
doc/test.qbk
View File

@@ -45,6 +45,7 @@
[def __BOOST_TEST_NO_LIB__ [link boost_test.utf_reference.link_references.link_boost_test_no_lib `BOOST_TEST_NO_LIB`]]
[def __BOOST_TEST_NO_MAIN__ [link boost_test.utf_reference.link_references.link_boost_test_no_main `BOOST_TEST_NO_MAIN`]]
[def __BOOST_TEST_MODULE__ [link boost_test.utf_reference.link_references.link_boost_test_module_macro `BOOST_TEST_MODULE`]]
[def __BOOST_TEST_GLOBAL_CONFIGURATION__ [link boost_test.utf_reference.link_references.link_boost_test_global_configuration `BOOST_TEST_GLOBAL_CONFIGURATION`]]
[def __BOOST_TEST_CHECKPOINT__ [link boost_test.utf_reference.testout_reference.test_output_macro_checkpoint `BOOST_TEST_CHECKPOINT`]]
[def __BOOST_TEST_PASSPOINT__ [link boost_test.utf_reference.testout_reference.test_output_macro_passpoint `BOOST_TEST_PASSPOINT`]]
@@ -116,7 +117,8 @@
[/ fixtures]
[def __BOOST_FIXTURE_TEST_CASE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_case_fixture `BOOST_FIXTURE_TEST_CASE`]]
[def __BOOST_FIXTURE_TEST_SUITE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_suite_fixture `BOOST_FIXTURE_TEST_SUITE`]]
[def __BOOST_GLOBAL_FIXTURE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_global_fixture `BOOST_GLOBAL_FIXTURE`]]
[def __BOOST_GLOBAL_FIXTURE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_global_fixture `BOOST_GLOBAL_FIXTURE`]]
[def __BOOST_TEST_GLOBAL_FIXTURE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_global_fixture `BOOST_TEST_GLOBAL_FIXTURE`]]
[/ log]
[def __BOOST_TEST_LOG_LEVEL__ [link boost_test.utf_reference.rt_param_reference.log_level `BOOST_TEST_LOG_LEVEL`]] [/ this is an environment variable]

143
doc/test_organization/fixtures.qbk
View File

@@ -1,5 +1,5 @@
[/
/ Copyright (c) 2003 Boost.Test contributors
/ Copyright (c) 2003 Boost.Test contributors
/
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
@@ -8,7 +8,7 @@
[section:fixtures Fixtures]
In general terms a test fixture or test context is the collection of one or more of the following items, required
to perform the test:
* preconditions
* particular states of tested units
* necessary cleanup procedures
@@ -22,19 +22,19 @@ has become even more popular. Single XP adopted test modules may contain hundred
many requiring very similar test setup/cleanup. This is the problem that the test fixture is designed to solve.
In practice a test fixture usually is a combination of `setup` and `teardown` functions, associated with test case.
The former serves the purposes of test setup. The later is dedicated to the cleanup tasks. Ideally we'd like for a
test module author to be able to define variables used in fixtures on the stack and, at the same time, to refer to
The former serves the purposes of test setup. The later is dedicated to the cleanup tasks. Ideally we'd like for a
test module author to be able to define variables used in fixtures on the stack and, at the same time, to refer to
them directly in a test case.
It's important to understand that C++ provides a way to implement a straightforward test fixture solution
that almost satisfies our requirements without any extra support from the test framework. Here is how simple test
that almost satisfies our requirements without any extra support from the test framework. Here is how simple test
module with such a fixture may look like:
``
struct MyFixture {
MyFixture() { i = new int; *i = 0 }
~MyFixture() { delete i; }
int* i;
};
@@ -51,7 +51,6 @@ __BOOST_AUTO_TEST_CASE__( test_case2 )
}
``
This is a generic solution that can be used to implement any kind of shared setup or cleanup procedure. Still
there are several more or less minor practical issues with this pure C++ based fixtures solution:
@@ -60,18 +59,18 @@ there are several more or less minor practical issues with this pure C++ based f
* There is no place to execute a ['global] fixture, which performs ['global] setup/cleanup
procedures before and after testing.
The __UTF__ lets you define a fixture according to [link boost_test.tests_organization.fixtures.models several generic interfaces],
The __UTF__ lets you define a fixture according to [link boost_test.tests_organization.fixtures.models several generic interfaces],
and thus helps you with following tasks:
* define shared setup/teardown procedures for a single or group of test cases
* define setup/teardown procedures which are performed once per test suite
* define global setup/teardown procedures which are performed once per test module
* define shared setup/teardown procedures for a single or group of test cases
* define setup/teardown procedures which are performed once per test suite
* define [link boost_test.tests_organization.fixtures.global global setup/teardown] procedures which are performed once per test module
[/ ###################################################################################### ]
[section:models Fixture models]
Several fixture interfaces are supported by the __UTF__. The choice of the interface depends
mainly on the usage of the fixture.
mainly on the usage of the fixture.
[h3 Fixture class model]
The __UTF__ defines the generic fixture class model as follows:
@@ -84,33 +83,57 @@ struct <fixture-name>{
``
In other words a fixture is expected to be implemented as a class where the class constructor serves as a `setup`
method and class destructor serves as `teardown` method. The __UTF__ opted to avoid explicit names in fixture
interface for `setup` and `teardown` methods, since it is considered most natural in C++ for tasks similar to RAII and
coincides with the pure C++ solution discussed above.
method and class destructor serves as `teardown` method.
The class model above has some limitations though:
* it is not possible to have exceptions in the teardown function, especially any test assertions that aborts the
current test case is not possible (as those use exceptions)
* it is sometimes more natural to use the constructor/destructor to perform the necessary resource allocation/release
of the fixture, and that will be consumed in the test cases, and check for the proper state of the fixture in separate functions.
Those checks are the pre-conditions for the test case to run, and the post-conditions that should be met after the test case
has been running.
This is why the __UTF__ also supports (Boost 1.65 on) optional `setup` and/or `teardown` functions as follow:
``
struct <fixture-name>{
<fixture-name>(); // ctor
~<fixture-name>(); // dtor
void setup(); // setup, optional
void teardown(); // teardown, optional
};
``
[note As mentioned, the declaration/implementation of the `setup` and `teardown` are optional:
the __UTF__ will check the existence of those and will call them adequately. However in C++98,
it is not possible to detect those declaration in case those are inherited (it works fine for
compiler supporting `auto` and `decltype`).
]
This model is expected from fixtures used with __BOOST_FIXTURE_TEST_CASE__ and __BOOST_FIXTURE_TEST_SUITE__.
[caution The above interface prevents you from reporting errors in the `teardown` procedure using an exception. It does make
sense though: if somehow more than one fixture is assigned to a test unit (e.g. using __decorator_fixture__ decorator),
you want all `teardown` procedures to run, even if some may experience problems.
]
[h3 Flexible models]
In addition to __BOOST_FIXTURE_TEST_CASE__ and __BOOST_FIXTURE_TEST_SUITE__ the __UTF__ allows to associate fixture with
test unit using the decorator __decorator_fixture__. This decorator supports additional models for declaring
In addition to __BOOST_FIXTURE_TEST_CASE__ and __BOOST_FIXTURE_TEST_SUITE__ the __UTF__ allows to associate fixture with
test unit using the decorator __decorator_fixture__. This decorator supports additional models for declaring
the `setup` and `teardown`:
* a fixture defined according to the fixture class model above
* a fixture defined according to the extended fixture class model, which allows for the fixture constructor to
* a fixture defined according to the extended fixture class model, which allows for the fixture constructor to
takes one argument. For example:
struct Fx
{
std::string s;
Fx(std::string s = "") : s(s)
{ BOOST_TEST_MESSAGE("set up " << s); }
~Fx() { BOOST_TEST_MESSAGE("tear down " << s); }
Fx(std::string s_ = "") : s(s_)
{ BOOST_TEST_MESSAGE("ctor " << s); }
void setup()
{ BOOST_TEST_MESSAGE("optional setup " << s); }
void setup()
{ BOOST_TEST_MESSAGE("optional teardown " << s); }
~Fx()
{ BOOST_TEST_MESSAGE("dtor " << s); }
};
* a fixture defined as a pair of free functions for the `setup` and `teardown` (latter optional)
@@ -127,7 +150,7 @@ For complete example of test module which uses these models please check decorat
[/ ###################################################################################### ]
[section:case Test case fixture]
A /test case fixture/ is a fixture consumed by a test case: the fixture `setup` is called before the test case executes,
and the fixture `teardown` is called after the test case finished its execution, independently from its execution state.
and the fixture `teardown` is called after the test case finished its execution, independently from its execution state.
The __UTF__ provides several ways of defining fixtures for test-cases, each of which having their properties:
@@ -140,30 +163,30 @@ The __UTF__ provides several ways of defining fixtures for test-cases, each of w
The following two methods are available for declaring a fixture attached to one particular test case:
* the use of the macro __BOOST_FIXTURE_TEST_CASE__ in place of __BOOST_AUTO_TEST_CASE__, which let access to the members of the fixture
* the use of the decorator __decorator_fixture__, which does not let access to the members but enables
the definition of several fixtures for one test case.
* the use of the decorator __decorator_fixture__, which does not let access to the members but enables
the definition of several fixtures for one test case.
[/ ------------------------------------------------------------------------ ]
[#test_case_fixture_macro][h4 Fixture with `BOOST_FIXTURE_TEST_CASE`]
`BOOST_FIXTURE_TEST_CASE` serves as a test case declaration with a fixture, and is meant be used in place of
`BOOST_FIXTURE_TEST_CASE` serves as a test case declaration with a fixture, and is meant be used in place of
the test case declaration with __BOOST_AUTO_TEST_CASE__:
``
BOOST_FIXTURE_TEST_CASE(test_case_name, fixture_name);
``
The only difference from the macro __BOOST_AUTO_TEST_CASE__ is the presence of an extra argument `fixture_name`.
The only difference from the macro __BOOST_AUTO_TEST_CASE__ is the presence of an extra argument `fixture_name`.
The public and protected members of the fixture are directly accessible from the test case body. Only
one fixture can be attached to a test-case [footnote it is still possible to define a class inheriting from several
one fixture can be attached to a test-case [footnote it is still possible to define a class inheriting from several
fixtures, that will act as a proxy fixture.].
[note You can't access private members of fixture, but then why would you make anything private?]
[bt_example example18..Per test case fixture..run-fail]
In this example only `test_case1` and `test_case2` have fixture `F` assigned.
You still need to refer to the fixture name in every test case. [link test_case_fixture_subtree This] section
In this example only `test_case1` and `test_case2` have fixture `F` assigned.
You still need to refer to the fixture name in every test case. [link test_case_fixture_subtree This] section
explains how a same fixture can be declared for a subtree under a test suite.
[/ ------------------------------------------------------------------------ ]
@@ -173,7 +196,7 @@ By using the decorator __decorator_fixture__, it is possible to:
* attach several fixtures to a unique test case
* use a flexible fixture interface (see [link boost_test.tests_organization.fixtures.models here])
[note Using the decorator approach, it is not possible to access the members of the fixture (in case the fixture is implemented
[note Using the decorator approach, it is not possible to access the members of the fixture (in case the fixture is implemented
as a class)]
@@ -196,7 +219,7 @@ of the fixture, but also do not need to refer to the fixture name in test case d
the same fixture automatically.
[tip If necessary you can reset the fixture for a particular test case using the macro
__BOOST_FIXTURE_TEST_CASE__. Similarly you can reset the fixture for a particular sub
__BOOST_FIXTURE_TEST_CASE__. Similarly you can reset the fixture for a particular sub
test suite using __BOOST_FIXTURE_TEST_SUITE__.
]
@@ -207,19 +230,18 @@ the same fixture automatically.
[bt_example fixture_02..Test suite level fixture..run]
[caution The fixture constructor and destructor is called for each test cases (the state of the
[caution The fixture constructor/setup and teardown/destructor is called for each test cases (the state of the
fixture is not shared among the test cases).]
[endsect] [/ per test case]
[/ ###################################################################################### ]
[section:per_test_suite_fixture Test suite entry/exit fixture]
It is possible to define a test suite entry/exit fixture, so that the `setup` function is called only once upon entering
the test suite, prior to running any of its test cases; and similarly the `teardown` function is also called only once
upon the test suite exit, after all the enclosed test cases have been run. This is facilitated by the
It is possible to define a test suite entry/exit fixture, so that the `setup` function is called only once upon entering
the test suite, prior to running any of its test cases.
Similarly the `teardown` function is also called only once
upon the test suite exit, after all the enclosed test cases have been run. This is facilitated by the
/decorator/ __decorator_fixture__.
[bt_example fixture_03..Test suite entry/exit fixture..run]
@@ -233,38 +255,43 @@ In case of this fixture type, however, it is not possible to access any members
[/ ###################################################################################### ]
[section:global Global fixture]
Any global initialization that needs to be performed every time testing begins or a global cleanup that is to be
performed once testing is finished is called a global fixture. The __UTF__ global fixture design is based on the
Any global initialization that needs to be performed before any test begins, or a cleanup that is to be
performed after all tests are finished is called a /global fixture/. A global fixture is equivalent to a
[link boost_test.tests_organization.fixtures.per_test_suite_fixture test-suite
entry/exit] fixture (executed once), where in this case the test-suite is the
[link boost_test.tests_organization.test_suite.master_test_suite master test suite].
The __UTF__ global fixture design is based on the
[link boost_test.tests_organization.fixtures.models generic test class fixture model]. The global
fixture design allows any number of global fixtures to be defined in any test file that constitutes a test module.
Though some initialization can be implemented in the test module initialization function, there are several
Though some initialization can be implemented in the test module initialization function, there are several
reasons to prefer the global fixture approach:
* There is no place for cleanup/`teardown` operations in the initialization function.
* Unlike the initialization function, the global fixture `setup` method invocation is guarded by the execution
monitor. That means that all uncaught errors that occur during initialization are properly reported.
* There is no place for `cleanup`/`teardown` operations in the initialization function.
* Unlike the initialization function, the global fixture construction, `setup` and `teardown` methods invocation are guarded by the
execution monitor. That means that all uncaught errors that occur during initialization are properly reported.
* Any number of different global fixtures can be defined, which allows you to split initialization code by
category.
* The fixture allows you to place matching `setup`/`teardown` code in close vicinity in your test module code.
* If the whole test tree is constructed automatically the initialization function is empty and auto-generated by
the __UTF__. To introduce the initialization function can be more work than the use of a global fixture facility,
* If the whole test tree is constructed automatically, the initialization function is empty and auto-generated by
the __UTF__. Introducing the initialization function can be more work than using the global fixture facility,
while global fixture is more to the point.
* Since all fixtures follow the same generic model you can easily switch from local per test case fixtures to
the global one.
* If you are using the interactive test runner (non-supplied yet) global test fixtures are applied to every run,
while an initialization function is executed only once during a test module startup (just make sure that
it's what you really want).
To define a global test module fixture you need to implement a class that matched generic fixture model and
passed it as an argument to the macro __BOOST_GLOBAL_FIXTURE__.
To define a global test module fixture you need:
# to implement a class that matches the
[link boost_test.tests_organization.fixtures.models fixture model]
# and to pass the class as an argument to the macro __BOOST_TEST_GLOBAL_FIXTURE__.
``
BOOST_GLOBAL_FIXTURE(fixture_name);
BOOST_TEST_GLOBAL_FIXTURE( fixture_name );
``
The statement, that performs global fixture definition, has to reside at a test file scope.
[bt_example example20..Global fixture..run]
[bt_example fixture_04..Global fixture..run-fail]
[endsect] [/section Global fixtures]

32
doc/test_organization/test_suites.qbk
View File

@@ -1,13 +1,13 @@
[/
/ Copyright (c) 2003 Boost.Test contributors
/ Copyright (c) 2003 Boost.Test contributors
/
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
/]
[section:test_suite Test suite]
If you consider test cases as leaves on the test tree, the test suite can be considered as branch and the master
test suite as a root. Unlike real trees though, our tree in many cases consists only of leaves attached
If you consider test cases as leaves on the test tree, the test suite can be considered as branch and the /master
test suite/ as the /root/. Unlike real trees though, our tree in many cases consists only of leaves attached
directly to the root. This is common for all test cases to reside directly in the master test suite. If you do
want to construct a hierarchical test suite structure the __UTF__ provides both manual and automated
test suite creation and registration facilities:
@@ -15,20 +15,20 @@ test suite creation and registration facilities:
# Test suite with automated registration
# Manually registered test suite
In addition the __UTF__ presents a notion of the
[link boost_test.tests_organization.test_suite.master_test_suite Master Test Suite].
The most important reason to learn about this component is that it provides an ability to access
In addition the __UTF__ presents a notion of the
[link boost_test.tests_organization.test_suite.master_test_suite Master Test Suite].
The most important reason to learn about this component is that it provides an ability to access
command line arguments supplied to a test module.
[#ref_BOOST_AUTO_TEST_SUITE][h3 Automated registration]
The solution the __UTF__ presents for automated test suite creation and registration is designed to facilitate
multiple points of definition, arbitrary test suites depth and smooth integration with automated test case creation
and registration. This facility should significantly simplify a test tree construction process in comparison with
multiple points of definition, arbitrary test suites depth and smooth integration with automated test case creation
and registration. This facility should significantly simplify a test tree construction process in comparison with
manual explicit registration case.
The implementation is based on the order of file scope variables definitions within a single compilation unit.
The semantic of this facility is very similar to the namespace feature of C++, including support for test suite
The semantic of this facility is very similar to the namespace feature of C++, including support for test suite
extension. To start test suite use the macro __BOOST_AUTO_TEST_SUITE__. To end test suite use the macro
__BOOST_AUTO_TEST_SUITE_END__. The same test suite can be restarted multiple times inside the same test file or in a
different test files. In a result all test units will be part of the same test suite in a constructed test tree.
@@ -57,9 +57,9 @@ you can see from the output both `test_case1` and `test_case2` reside in the sam
[bt_example example53..Test suite extension using automated registration facility..run-fail]
[h3 Test suites with manual registration]
To create a test suite manually you need to
To create a test suite manually you need to
# create an instance of [classref boost::unit_test::test_suite] class,
# create an instance of [classref boost::unit_test::test_suite] class,
# register it in test tree, and
# populate it with test cases (or lower level test suites).
@@ -87,10 +87,10 @@ default no errors are expected.
The third optional parameter - `timeout` - defines the timeout value for the test unit. As of now the __UTF__
isn't able to set a timeout for the test suite execution, so this parameter makes sense only for test case
registration. By default no timeout is set. See the method
[memberref boost::execution_monitor::execute] for more details about the timeout value. [warning is the reference
[memberref boost::execution_monitor::execute] for more details about the timeout value. [warning is the reference
good? It looks to me that [memberref boost::unit_test::test_suite::add] is better]
To register group of test units in one function call, the [classref boost::unit_test::test_suite `test_suite`] class provides another
To register group of test units in one function call, the [classref boost::unit_test::test_suite `test_suite`] class provides another
[memberref boost::unit_test::test_suite::add `add`] interface covered in the advanced section of this documentation.
@@ -123,8 +123,8 @@ The example below creates a test tree, which can be represented by the following
[section:master_test_suite Master Test Suite]
As defined in introduction section the master test suite is a root node of a test tree. Each test module built
with the __UTF__ always has the master test suite defined. The __UTF__ maintain the master test suite instance
As defined in introduction section the master test suite is the *root* node of the test tree. Each test module built
with the __UTF__ always has the (unique) master test suite defined. The __UTF__ maintain the master test suite instance
internally. All other test units are registered as direct or indirect children of the master test suite.
``
@@ -192,7 +192,7 @@ macro value becomes the name of the master test suite. The name may include spac
[bt_example example15..Naming master test suite explicitly in the test module initialization function..run]
Without the __BOOST_TEST_MAIN__ and the __BOOST_TEST_MODULE__ flags defined, the test module initialization
Without the __BOOST_TEST_MAIN__ and the __BOOST_TEST_MODULE__ flags defined, the test module initialization
function has to be manually implemented. The master test suite name can be reset at any point within this function.
[endsect] [/ command line interface]

37
doc/test_organization/testorg_reference.qbk
View File

@@ -166,13 +166,20 @@ See [link boost_test.tests_organization.fixtures.case here] for more details.
[/-----------------------------------------------------------------]
[section:test_org_boost_global_fixture `BOOST_GLOBAL_FIXTURE`]
Declares and registers a global fixture. The global fixture is called before any of the test case in the test tree is executed.
This macro is deprecated in favor of __BOOST_TEST_GLOBAL_FIXTURE__ and __BOOST_TEST_GLOBAL_CONFIGURATION__.
[endsect] [/section:test_org_boost_test_case_fixture]
[/-----------------------------------------------------------------]
[section:test_org_boost_test_global_fixture `BOOST_TEST_GLOBAL_FIXTURE`]
Declares and registers a global fixture. The global fixture acts exactly as a suite fixture attached to the
[link boost_test.tests_organization.test_suite.master_test_suite master test suite],
and is called before any of the test case in the test tree is executed.
The class implementing the fixture should have the appropriate [link boost_test.tests_organization.fixtures.models interface].
As any fixture, it is possible to have test assertions in the global fixture.
See [link boost_test.tests_organization.fixtures.global here] for more details.
[endsect] [/section:test_org_boost_test_case_fixture]
[endsect] [/section:test_org_boost_test_global_fixture]
@@ -257,12 +264,24 @@ template <typename Fx, typename Arg>
Decorator `fixture` specifies a pair of functions (like `set_up` and `tear_down`) to be called before and after the
corresponding test unit. At the suite level the `set_up` function is called once -- before the suite execution starts
and `tear_down` function is called once -- after the suite execution ends. It comes in three forms. First expects two
functions for set-up and tear-down (the second one can be skipped). The second expects a `DefaultConstructible` class.
Its default constructor will be used as set-up function and its destructor as a tear-down function. Third requires a
class with one-argument public constructor. Argument `arg` is forwarded to the constructor and this is the set-up
function, its destructor is the tear-down function. There is no way to get access to the members of these fixtures from
within the test case or test suite.
-- and `tear_down` function is called once -- after the suite execution ends. It comes in three forms.
First expects two
functions for set-up and tear-down (the second one can be skipped).
The second expects a `DefaultConstructible` class.
Its default constructor will be used as set-up function and its destructor as a tear-down function.
The third form requires a
class with one-argument public constructor. Argument `arg` is forwarded to the constructor.
For the second and third form, the framework detects if there is a `setup` and/or `teardown` function implemented in the class,
with the same declaration as described in the [link boost_test.tests_organization.fixtures.models fixture model].
If those member function are declared, they will be called right after construction and just
before destruction respectively.
[note There is no way to get access to the members of these fixtures from
within the test case or test suite.]
[bt_example decorator_12..decorator fixture..run]

2
doc/test_organization/testorg_summary.qbk
View File

@@ -88,7 +88,7 @@
[Declares a fixture for a test suite (the setup/teardown is called for each test of the test suite)]
]
[
[__BOOST_GLOBAL_FIXTURE__]
[__BOOST_TEST_GLOBAL_FIXTURE__]
[Declares a fixture globally to the test module]
]

9
doc/test_output/logger_api.qbk
View File

@@ -9,10 +9,15 @@
While many test log configuration tasks can be performed at runtime using predefined framework parameters, the
__UTF__ provides a compile time interface as well. The interface gives you full power over what, where and how to
log. The interface is provided by singleton class [classref boost::unit_test::unit_test_log_t] and is
log. The interface of the logger is provided by singleton class [classref boost::unit_test::unit_test_log_t] and is
accessible through local file scope reference to single instance of this class
``boost::unit_test::unit_test_log``
``
boost::unit_test::unit_test_log
``
In order to install customization of the logger, the __UTF__ provides the __BOOST_TEST_GLOBAL_CONFIGURATION__ facility
that acts in a similar fashion to a global fixture.
[/ ------------------------------------------------------------------------------------------------ ]

10
example/unit_test_example_09_1.cpp
View File

@@ -1,6 +1,6 @@
// (C) Copyright Gennadiy Rozental 2005-2014.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
@@ -14,13 +14,13 @@
//____________________________________________________________________________//
struct MyConfig {
MyConfig() { std::cout << "global setup part1\n"; }
~MyConfig() { std::cout << "global teardown part1\n"; }
struct MyConfig {
MyConfig() { std::cout << "global setup part1\n"; }
~MyConfig() { std::cout << "global teardown part1\n"; }
};
// structure MyConfig is used as a global fixture - it's invoked pre and post any testing is performed
BOOST_GLOBAL_FIXTURE( MyConfig );
BOOST_TEST_GLOBAL_FIXTURE( MyConfig );
//____________________________________________________________________________//

10
example/unit_test_example_09_2.cpp
View File

@@ -1,6 +1,6 @@
// (C) Copyright Gennadiy Rozental 2005-2014.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
@@ -14,13 +14,13 @@
//____________________________________________________________________________//
struct MyConfig2 {
MyConfig2() { std::cout << "global setup part2\n"; }
~MyConfig2() { std::cout << "global teardown part2\n"; }
struct MyConfig2 {
MyConfig2() { std::cout << "global setup part2\n"; }
~MyConfig2() { std::cout << "global teardown part2\n"; }
};
// structure MyConfig2 is used as a global fixture. You could have any number of global fxtures
BOOST_GLOBAL_FIXTURE( MyConfig2 );
BOOST_TEST_GLOBAL_FIXTURE( MyConfig2 );
//____________________________________________________________________________//

1
include/boost/test/detail/fwd_decl.hpp
View File

@@ -26,6 +26,7 @@ class master_test_suite_t;
class test_tree_visitor;
class test_observer;
class test_unit_fixture;
// singletons
class unit_test_monitor_t;

15
include/boost/test/detail/global_typedef.hpp
View File

@@ -71,6 +71,21 @@ test_id_2_unit_type( test_unit_id id )
return (id & 0xFFFF0000) != 0 ? TUT_CASE : TUT_SUITE;
}
//! Helper class for restoring the current test unit ID in a RAII manner
struct test_unit_id_restore {
test_unit_id_restore(test_unit_id& to_restore_, test_unit_id new_value)
: to_restore(to_restore_)
, bkup(to_restore_) {
to_restore = new_value;
}
~test_unit_id_restore() {
to_restore = bkup;
}
private:
test_unit_id& to_restore;
test_unit_id bkup;
};
//____________________________________________________________________________//
} // namespace ut_detail

1
include/boost/test/execution_monitor.hpp
View File

@@ -270,6 +270,7 @@ public:
/// Simple model for the location of failure in a source code
struct BOOST_TEST_DECL location {
explicit location( char const* file_name = 0, size_t line_num = 0, char const* func = 0 );
explicit location( const_string file_name, size_t line_num = 0, char const* func = 0 );
const_string m_file_name; ///< File name
size_t m_line_num; ///< Line number

31
include/boost/test/framework.hpp
View File

@@ -74,7 +74,7 @@ BOOST_TEST_DECL bool test_in_progress();
/// This function shuts down the framework and clears up its mono-state.
///
/// It needs to be at the very end of test module execution
BOOST_TEST_DECL void shutdown();
BOOST_TEST_DECL void shutdown();
/// @}
/// @name Test unit registration
@@ -132,12 +132,28 @@ BOOST_TEST_DECL void clear();
/// @param[in] to test observer object to add
BOOST_TEST_DECL void register_observer( test_observer& to );
/// Excldes the observer object form the framework's list of test observers
/// Excludes the observer object form the framework's list of test observers
/// @param[in] to test observer object to exclude
BOOST_TEST_DECL void deregister_observer( test_observer& to );
/// @}
/// @name Global fixtures registration
/// @{
/// Adds a new global fixture to be setup before any other tests starts and tore down after
/// any other tests finished.
/// Test unit fixture lifetime should exceed the testing execution timeframe
/// @param[in] tuf fixture to add
BOOST_TEST_DECL void register_global_fixture( test_unit_fixture& tuf );
/// Removes a test global fixture from the framework
///
/// Test unit fixture lifetime should exceed the testing execution timeframe
/// @param[in] tuf fixture to remove
BOOST_TEST_DECL void deregister_global_fixture( test_unit_fixture& tuf );
/// @}
/// @name Assertion/uncaught exception context support
/// @{
/// Context accessor
@@ -178,6 +194,15 @@ BOOST_TEST_DECL context_generator get_context();
/// @returns a reference the master test suite instance
BOOST_TEST_DECL master_test_suite_t& master_test_suite();
/// This function provides an access to the test unit currently being executed.
/// The difference with current_test_case is about the time between a test-suite
/// is being set up or torn down (fixtures) and when the test-cases of that suite start.
/// This function is only valid during test execution phase.
/// @see current_test_case_id, current_test_case
BOOST_TEST_DECL test_unit const& current_test_unit();
/// This function provides an access to the test case currently being executed.
/// This function is only valid during test execution phase.
@@ -231,6 +256,8 @@ BOOST_TEST_DECL void assertion_result( unit_test::assertion_resul
BOOST_TEST_DECL void exception_caught( execution_exception const& );
/// Reports aborted test unit to all test observers
BOOST_TEST_DECL void test_unit_aborted( test_unit const& );
/// Reports aborted test module to all test observers
BOOST_TEST_DECL void test_aborted( );
/// @}
namespace impl {

2
include/boost/test/impl/compiler_log_formatter.ipp
View File

@@ -52,7 +52,7 @@ namespace {
std::string
test_phase_identifier()
{
return framework::test_in_progress() ? framework::current_test_case().full_name() : std::string( "Test setup" );
return framework::test_in_progress() ? framework::current_test_unit().full_name() : std::string( "Test setup" );
}
} // local namespace

6
include/boost/test/impl/execution_monitor.ipp
View File

@@ -1342,6 +1342,12 @@ execution_exception::location::location( char const* file_name, size_t line_num,
, m_function( func )
{}
execution_exception::location::location(const_string file_name, size_t line_num, char const* func )
: m_file_name( file_name )
, m_line_num( line_num )
, m_function( func )
{}
//____________________________________________________________________________//
// ************************************************************************** //

184
include/boost/test/impl/framework.ipp
View File

@@ -27,6 +27,7 @@
#include <boost/test/results_collector.hpp>
#include <boost/test/progress_monitor.hpp>
#include <boost/test/results_reporter.hpp>
#include <boost/test/test_framework_init_observer.hpp>
#include <boost/test/tree/observer.hpp>
#include <boost/test/tree/test_unit.hpp>
@@ -71,6 +72,7 @@ namespace std { using ::time; using ::srand; }
namespace boost {
namespace unit_test {
namespace framework {
namespace impl {
// ************************************************************************** //
@@ -457,6 +459,27 @@ void random_shuffle( RandomIt first, RandomIt last, RandomFunc &r )
#endif
// A simple handle for registering the global fixtures to the master test suite
// without deleting an existing static object (the global fixture itself) when the program
// terminates (shared_ptr).
class global_fixture_handle : public test_unit_fixture {
public:
global_fixture_handle(test_unit_fixture* fixture) : m_global_fixture(fixture) {}
~global_fixture_handle() {}
virtual void setup() {
m_global_fixture->setup();
}
virtual void teardown() {
m_global_fixture->teardown();
}
private:
test_unit_fixture* m_global_fixture;
};
} // namespace impl
// ************************************************************************** //
@@ -468,7 +491,7 @@ unsigned const TIMEOUT_EXCEEDED = static_cast<unsigned>( -1 );
class state {
public:
state()
: m_curr_test_case( INV_TEST_UNIT_ID )
: m_curr_test_unit( INV_TEST_UNIT_ID )
, m_next_test_case_id( MIN_TEST_CASE_ID )
, m_next_test_suite_id( MIN_TEST_SUITE_ID )
, m_test_in_progress( false )
@@ -625,7 +648,7 @@ public:
}
};
// Executed the test tree with the root at specified test unit
// Executes the test tree with the root at specified test unit
execution_result execute_test_tree( test_unit_id tu_id,
unsigned timeout = 0,
random_generator_helper const * const p_random_generator = 0)
@@ -664,9 +687,15 @@ public:
// 30. Execute setup fixtures if any; any failure here leads to test unit abortion
BOOST_TEST_FOREACH( test_unit_fixture_ptr, F, tu.p_fixtures.get() ) {
ut_detail::test_unit_id_restore restore_current_test_unit(m_curr_test_unit, tu.p_id);
result = unit_test_monitor.execute_and_translate( boost::bind( &test_unit_fixture::setup, F ) );
if( result != unit_test_monitor_t::test_ok )
break;
test_results const& test_rslt = unit_test::results_collector.results( m_curr_test_unit );
if( test_rslt.aborted() ) {
result = unit_test_monitor_t::precondition_failure;
break;
}
}
// This is the time we are going to spend executing the test unit
@@ -736,8 +765,7 @@ public:
m_context_idx = 0;
// setup current test case
test_unit_id bkup = m_curr_test_case;
m_curr_test_case = tc.p_id;
ut_detail::test_unit_id_restore restore_current_test_unit(m_curr_test_unit, tc.p_id);
// execute the test case body
result = unit_test_monitor.execute_and_translate( tc.p_test_func, timeout );
@@ -746,8 +774,7 @@ public:
// cleanup leftover context
m_context.clear();
// restore state and abort if necessary
m_curr_test_case = bkup;
// restore state (scope exit) and abort if necessary
}
}
@@ -755,6 +782,7 @@ public:
if( !unit_test_monitor.is_critical_error( result ) ) {
// execute teardown fixtures if any in reverse order
BOOST_TEST_REVERSE_FOREACH( test_unit_fixture_ptr, F, tu.p_fixtures.get() ) {
ut_detail::test_unit_id_restore restore_current_test_unit(m_curr_test_unit, tu.p_id);
result = (std::min)( result, unit_test_monitor.execute_and_translate( boost::bind( &test_unit_fixture::teardown, F ), 0 ) );
if( unit_test_monitor.is_critical_error( result ) )
@@ -813,7 +841,7 @@ public:
master_test_suite_t* m_master_test_suite;
std::vector<test_suite*> m_auto_test_suites;
test_unit_id m_curr_test_case;
test_unit_id m_curr_test_unit;
test_unit_store m_test_units;
test_unit_id m_next_test_case_id;
@@ -825,6 +853,8 @@ public:
context_data m_context;
int m_context_idx;
std::set<test_unit_fixture*> m_global_fixtures;
boost::execution_monitor m_aux_em;
std::map<output_format, runtime_config::stream_holder> m_log_sinks;
@@ -1069,6 +1099,7 @@ init( init_unit_test_func init_func, int argc, char* argv[] )
// 40. Register default test observers
register_observer( results_collector );
register_observer( unit_test_log );
register_observer( framework_init_observer );
if( runtime_config::get<bool>( runtime_config::btrt_show_progress ) ) {
progress_monitor.set_stream( std::cout ); // defaults to stdout
@@ -1261,6 +1292,30 @@ deregister_observer( test_observer& to )
//____________________________________________________________________________//
// ************************************************************************** //
// ************** register_global_fixture ************** //
// ************************************************************************** //
void
register_global_fixture( test_unit_fixture& tuf )
{
impl::s_frk_state().m_global_fixtures.insert( &tuf );
}
//____________________________________________________________________________//
// ************************************************************************** //
// ************** deregister_global_fixture ************** //
// ************************************************************************** //
void
deregister_global_fixture( test_unit_fixture &tuf )
{
impl::s_frk_state().m_global_fixtures.erase( &tuf );
}
//____________________________________________________________________________//
// ************************************************************************** //
// ************** add_context ************** //
// ************************************************************************** //
@@ -1389,7 +1444,14 @@ current_auto_test_suite( test_suite* ts, bool push_or_pop )
test_case const&
current_test_case()
{
return get<test_case>( impl::s_frk_state().m_curr_test_case );
return get<test_case>( impl::s_frk_state().m_curr_test_unit );
}
test_unit const&
current_test_unit()
{
return *impl::s_frk_state().m_test_units[impl::s_frk_state().m_curr_test_unit];
}
//____________________________________________________________________________//
@@ -1397,7 +1459,7 @@ current_test_case()
test_unit_id
current_test_case_id()
{
return impl::s_frk_state().m_curr_test_case;
return impl::s_frk_state().m_curr_test_unit;
}
//____________________________________________________________________________//
@@ -1422,6 +1484,17 @@ get( test_unit_id id, test_unit_type t )
// ************** framework::run ************** //
// ************************************************************************** //
template <class Cont>
struct swap_on_delete {
swap_on_delete(Cont& c1, Cont& c2) : m_c1(c1), m_c2(c2){}
~swap_on_delete() {
m_c1.swap(m_c2);
}
Cont& m_c1;
Cont& m_c2;
};
void
run( test_unit_id id, bool continue_test )
{
@@ -1440,39 +1513,90 @@ run( test_unit_id id, bool continue_test )
bool was_in_progress = framework::test_in_progress();
bool call_start_finish = !continue_test || !was_in_progress;
impl::s_frk_state().m_test_in_progress = true;
bool init_ok = true;
const_string setup_error;
if( call_start_finish ) {
// indicates the framework that no test is in progress now if observers need to be notified
impl::s_frk_state().m_test_in_progress = false;
// unit_test::framework_init_observer will get cleared first
BOOST_TEST_FOREACH( test_observer*, to, impl::s_frk_state().m_observers ) {
BOOST_TEST_I_TRY {
impl::s_frk_state().m_aux_em.vexecute( boost::bind( &test_observer::test_start, to, tcc.p_count ) );
ut_detail::test_unit_id_restore restore_current_test_unit(impl::s_frk_state().m_curr_test_unit, id);
unit_test_monitor_t::error_level result = unit_test_monitor.execute_and_translate( boost::bind( &test_observer::test_start, to, tcc.p_count ) );
if( init_ok ) {
if( result != unit_test_monitor_t::test_ok ) {
init_ok = false;
}
else {
if( unit_test::framework_init_observer.has_failed() ) {
init_ok = false;
}
}
}
}
BOOST_TEST_I_CATCH( execution_exception, ex ) {
BOOST_TEST_SETUP_ASSERT( false, ex.what() );
if( init_ok ) {
// log only the first error
init_ok = false;
setup_error = ex.what();
}
// break; // we should continue otherwise loggers may have improper structure (XML start missing for instance)
}
}
}
unsigned seed = runtime_config::get<unsigned>( runtime_config::btrt_random_seed );
switch( seed ) {
case 0:
break;
case 1:
seed = static_cast<unsigned>( std::rand() ^ std::time( 0 ) ); // better init using std::rand() ^ ...
default:
BOOST_TEST_FRAMEWORK_MESSAGE( "Test cases order is shuffled using seed: " << seed );
std::srand( seed );
if( init_ok ) {
// attaching the global fixtures to the main entry point
test_unit& entry_test_unit = framework::get( id, TUT_ANY );
std::vector<test_unit_fixture_ptr> v_saved_fixture(entry_test_unit.p_fixtures.value.begin(),
entry_test_unit.p_fixtures.value.end());
BOOST_TEST_FOREACH( test_unit_fixture*, tuf, impl::s_frk_state().m_global_fixtures ) {
entry_test_unit.p_fixtures.value.insert( entry_test_unit.p_fixtures.value.begin(),
test_unit_fixture_ptr(new impl::global_fixture_handle(tuf)) );
}
swap_on_delete< std::vector<test_unit_fixture_ptr> > raii_fixture(v_saved_fixture, entry_test_unit.p_fixtures.value);
// now work in progress
impl::s_frk_state().m_test_in_progress = true;
unsigned seed = runtime_config::get<unsigned>( runtime_config::btrt_random_seed );
switch( seed ) {
case 0:
break;
case 1:
seed = static_cast<unsigned>( std::rand() ^ std::time( 0 ) ); // better init using std::rand() ^ ...
default:
BOOST_TEST_FRAMEWORK_MESSAGE( "Test cases order is shuffled using seed: " << seed );
std::srand( seed );
}
// executing the test tree
impl::s_frk_state().execute_test_tree( id );
// removing previously added global fixtures: dtor raii_fixture
}
impl::s_frk_state().execute_test_tree( id );
impl::s_frk_state().m_test_in_progress = false;
unit_test::framework_init_observer.clear();
if( call_start_finish ) {
BOOST_TEST_REVERSE_FOREACH( test_observer*, to, impl::s_frk_state().m_observers )
// indicates the framework that no test is in progress anymore if observers need to be notified
// and this is a teardown, so assertions should not raise any exception otherwise an exception
// might be raised in a dtor of a global fixture
impl::s_frk_state().m_test_in_progress = false;
BOOST_TEST_REVERSE_FOREACH( test_observer*, to, impl::s_frk_state().m_observers ) {
ut_detail::test_unit_id_restore restore_current_test_unit(impl::s_frk_state().m_curr_test_unit, id);
to->test_finish();
}
}
impl::s_frk_state().m_test_in_progress = was_in_progress;
// propagates the init/teardown error if any
BOOST_TEST_SETUP_ASSERT( init_ok && !unit_test::framework_init_observer.has_failed(), setup_error );
}
//____________________________________________________________________________//
@@ -1522,6 +1646,18 @@ test_unit_aborted( test_unit const& tu )
to->test_unit_aborted( tu );
}
// ************************************************************************** //
// ************** test_aborted ************** //
// ************************************************************************** //
void
test_aborted( )
{
BOOST_TEST_FOREACH( test_observer*, to, impl::s_frk_state().m_observers )
to->test_aborted( );
}
//____________________________________________________________________________//
} // namespace framework

238
include/boost/test/impl/junit_log_formatter.ipp
View File

@@ -69,6 +69,12 @@ inline std::string tu_name_normalize(std::string full_name)
return full_name;
}
inline std::string tu_name_remove_newlines(std::string full_name)
{
full_name.erase(std::remove(full_name.begin(), full_name.end(), '\n'), full_name.end());
return full_name;
}
const_string file_basename(const_string filename) {
const_string path_sep( "\\/" );
@@ -96,6 +102,11 @@ junit_log_formatter::log_start( std::ostream& ostr, counter_t test_cases_amount)
//____________________________________________________________________________//
class junit_result_helper : public test_tree_visitor {
private:
typedef junit_impl::junit_log_helper::assertion_entry assertion_entry;
typedef std::vector< assertion_entry >::const_iterator vect_assertion_entry_citerator;
typedef std::list<std::string>::const_iterator list_str_citerator;
public:
explicit junit_result_helper(
std::ostream& stream,
@@ -111,10 +122,19 @@ public:
, m_display_build_info(display_build_info)
{ }
void add_log_entry(std::string const& entry_type,
test_case const& tc,
junit_impl::junit_log_helper::assertion_entry const& log) const
void add_log_entry(assertion_entry const& log) const
{
std::string entry_type;
if( log.log_entry == assertion_entry::log_entry_failure ) {
entry_type = "failure";
}
else if( log.log_entry == assertion_entry::log_entry_error ) {
entry_type = "error";
}
else {
return;
}
m_stream
<< "<" << entry_type
<< " message" << utils::attr_value() << log.logentry_message
@@ -159,18 +179,18 @@ public:
}
};
std::list<std::string> build_skipping_chain(test_case const & tc) const
std::list<std::string> build_skipping_chain(test_unit const & tu) const
{
// we enter here because we know that the tc has been skipped.
// either junit has not seen this tc, or it is indicated as disabled
assert(m_map_test.count(tc.p_id) == 0 || results_collector.results( tc.p_id ).p_skipped);
// we enter here because we know that the tu has been skipped.
// either junit has not seen this tu, or it is indicated as disabled
assert(m_map_test.count(tu.p_id) == 0 || results_collector.results( tu.p_id ).p_skipped);
std::list<std::string> out;
test_unit_id id(tc.p_id);
test_unit_id id(tu.p_id);
while( id != m_ts.p_id && id != INV_TEST_UNIT_ID) {
test_unit const& tu = boost::unit_test::framework::get( id, TUT_ANY );
out.push_back("- disabled test unit: '" + tu.full_name() + "'\n");
out.push_back("- disabled test unit: '" + tu_name_remove_newlines(tu.full_name()) + "'\n");
if(m_map_test.count(id) > 0)
{
// junit has seen the reason: this is enough for constructing the chain
@@ -188,9 +208,9 @@ public:
return out;
}
std::string get_class_name(test_case const & tc) const {
std::string get_class_name(test_unit const & tu_class) const {
std::string classname;
test_unit_id id(tc.p_parent_id);
test_unit_id id(tu_class.p_parent_id);
while( id != m_ts.p_id && id != INV_TEST_UNIT_ID ) {
test_unit const& tu = boost::unit_test::framework::get( id, TUT_ANY );
classname = tu_name_normalize(tu.p_name) + "." + classname;
@@ -205,39 +225,47 @@ public:
return classname;
}
void write_testcase_header(test_case const & tc,
test_results const *tr = 0) const
void write_testcase_header(test_unit const & tu,
test_results const *tr,
int nb_assertions) const
{
//
// test case header
std::string name;
std::string classname;
// total number of assertions
m_stream << "<testcase assertions" << utils::attr_value() << tr->p_assertions_passed + tr->p_assertions_failed;
if(tu.p_id == m_ts.p_id ) {
name = "boost_test";
}
else {
classname = get_class_name(tu);
name = tu_name_normalize(tu.p_name);
}
// class name
const std::string classname = get_class_name(tc);
if( tu.p_type == TUT_SUITE ) {
name += "-setup-teardown";
}
m_stream << "<testcase assertions" << utils::attr_value() << nb_assertions;
if(!classname.empty())
m_stream << " classname" << utils::attr_value() << classname;
// test case name and time taken
m_stream
<< " name" << utils::attr_value() << tu_name_normalize(tc.p_name)
<< " name" << utils::attr_value() << name
<< " time" << utils::attr_value() << double(tr->p_duration_microseconds) * 1E-6
<< ">" << std::endl;
}
void write_testcase_system_out(junit_impl::junit_log_helper const &detailed_log,
test_case const * tc,
bool skipped,
test_results const *tr = 0) const
test_unit const * tu,
bool skipped) const
{
// system-out + all info/messages, the object skips the empty entries
conditional_cdata_helper system_out_helper(m_stream, "system-out");
// indicate why the test has been skipped first
if( skipped ) {
std::list<std::string> skipping_decision_chain = build_skipping_chain(*tc);
for(std::list<std::string>::const_iterator it(skipping_decision_chain.begin()), ite(skipping_decision_chain.end());
std::list<std::string> skipping_decision_chain = build_skipping_chain(*tu);
for(list_str_citerator it(skipping_decision_chain.begin()), ite(skipping_decision_chain.end());
it != ite;
++it)
{
@@ -246,7 +274,7 @@ public:
}
// stdout
for(std::list<std::string>::const_iterator it(detailed_log.system_out.begin()), ite(detailed_log.system_out.end());
for(list_str_citerator it(detailed_log.system_out.begin()), ite(detailed_log.system_out.end());
it != ite;
++it)
{
@@ -254,25 +282,24 @@ public:
}
// warning/info message last
for(std::vector< junit_impl::junit_log_helper::assertion_entry >::const_iterator it(detailed_log.assertion_entries.begin());
for(vect_assertion_entry_citerator it(detailed_log.assertion_entries.begin());
it != detailed_log.assertion_entries.end();
++it)
{
if(it->log_entry != junit_impl::junit_log_helper::assertion_entry::log_entry_info)
if(it->log_entry != assertion_entry::log_entry_info)
continue;
system_out_helper(it->output);
}
}
void write_testcase_system_err(junit_impl::junit_log_helper const &detailed_log,
test_case const * tc,
test_results const *tr = 0) const
test_unit const * tu,
test_results const *tr) const
{
// system-err output + test case informations
bool has_failed = (tr != 0) ? !tr->passed() : false;
bool has_failed = (tr != 0) ? !tr->p_skipped && !tr->passed() : false;
if(!detailed_log.system_err.empty() || has_failed)
{
conditional_cdata_helper system_err_helper(m_stream, "system-err");
std::ostringstream o;
if(has_failed) {
o << "Failures detected in:" << std::endl;
@@ -281,58 +308,89 @@ public:
o << "ERROR STREAM:" << std::endl;
}
o << "- test case: " << tc->full_name() << std::endl;
if(!tc->p_description.value.empty())
o << " '" << tc->p_description << "'";
if(tu->p_type == TUT_SUITE) {
if( tu->p_id == m_ts.p_id ) {
o << " boost.test global setup/teardown" << std::endl;
} else {
o << "- test suite: " << tu_name_remove_newlines(tu->full_name()) << std::endl;
}
}
else {
o << "- test case: " << tu_name_remove_newlines(tu->full_name());
if(!tu->p_description.value.empty())
o << " '" << tu->p_description << "'";
o << std::endl
<< "- file: " << file_basename(tc->p_file_name) << std::endl
<< "- line: " << tc->p_line_num << std::endl
;
o << std::endl
<< "- file: " << file_basename(tu->p_file_name) << std::endl
<< "- line: " << tu->p_line_num << std::endl
;
}
if(!detailed_log.system_err.empty())
o << std::endl << "STDERR BEGIN: ------------" << std::endl;
system_err_helper(o.str());
for(std::list<std::string>::const_iterator it(detailed_log.system_err.begin()), ite(detailed_log.system_err.end());
for(list_str_citerator it(detailed_log.system_err.begin()), ite(detailed_log.system_err.end());
it != ite;
++it)
{
system_err_helper(*it);
o << *it;
}
if(!detailed_log.system_err.empty())
o << std::endl << "STDERR END ------------" << std::endl;
conditional_cdata_helper system_err_helper(m_stream, "system-err");
system_err_helper(o.str());
}
}
int get_nb_assertions(junit_impl::junit_log_helper const &detailed_log,
test_unit const & tu,
test_results const *tr) const {
int nb_assertions(-1);
if( tu.p_type == TUT_SUITE ) {
nb_assertions = 0;
for(vect_assertion_entry_citerator it(detailed_log.assertion_entries.begin());
it != detailed_log.assertion_entries.end();
++it)
{
if(it->log_entry != assertion_entry::log_entry_info)
nb_assertions++;
}
}
else {
nb_assertions = tr->p_assertions_passed + tr->p_assertions_failed;
}
return nb_assertions;
}
void output_detailed_logs(junit_impl::junit_log_helper const &detailed_log,
test_case const & tc,
test_unit const & tu,
bool skipped,
test_results const *tr = 0) const
test_results const *tr) const
{
write_testcase_header(tc, tr);
int nb_assertions = get_nb_assertions(detailed_log, tu, tr);
if(!nb_assertions && tu.p_type == TUT_SUITE)
return;
write_testcase_header(tu, tr, nb_assertions);
if( skipped ) {
m_stream << "<skipped/>" << std::endl;
}
else {
for(std::vector< junit_impl::junit_log_helper::assertion_entry >::const_iterator it(detailed_log.assertion_entries.begin());
for(vect_assertion_entry_citerator it(detailed_log.assertion_entries.begin());
it != detailed_log.assertion_entries.end();
++it)
{
if(it->log_entry == junit_impl::junit_log_helper::assertion_entry::log_entry_failure) {
add_log_entry("failure", tc, *it);
}
else if(it->log_entry == junit_impl::junit_log_helper::assertion_entry::log_entry_error) {
add_log_entry("error", tc, *it);
}
add_log_entry(*it);
}
}
write_testcase_system_out(detailed_log, &tc, skipped, tr);
write_testcase_system_err(detailed_log, &tc, tr);
write_testcase_system_out(detailed_log, &tu, skipped);
write_testcase_system_err(detailed_log, &tu, tr);
m_stream << "</testcase>" << std::endl;
}
@@ -353,35 +411,45 @@ public:
bool test_suite_start( test_suite const& ts )
{
// unique test suite, without s, nesting not supported in CI
if( m_ts.p_id != ts.p_id )
return true;
test_results const& tr = results_collector.results( ts.p_id );
m_stream << "<testsuite";
m_stream
// << "disabled=\"" << tr.p_test_cases_skipped << "\" "
<< " tests" << utils::attr_value() << tr.p_test_cases_passed
<< " skipped" << utils::attr_value() << tr.p_test_cases_skipped
<< " errors" << utils::attr_value() << tr.p_test_cases_aborted
<< " failures" << utils::attr_value() << tr.p_test_cases_failed
<< " id" << utils::attr_value() << m_id++
<< " name" << utils::attr_value() << tu_name_normalize(ts.p_name)
<< " time" << utils::attr_value() << (tr.p_duration_microseconds * 1E-6)
<< ">" << std::endl;
// unique test suite, without s, nesting not supported in CI
if( m_ts.p_id == ts.p_id ) {
m_stream << "<testsuite";
if(m_display_build_info)
{
m_stream << "<properties>" << std::endl;
m_stream << "<property name=\"platform\" value" << utils::attr_value() << BOOST_PLATFORM << std::endl;
m_stream << "<property name=\"compiler\" value" << utils::attr_value() << BOOST_COMPILER << std::endl;
m_stream << "<property name=\"stl\" value" << utils::attr_value() << BOOST_STDLIB << std::endl;
m_stream
// << "disabled=\"" << tr.p_test_cases_skipped << "\" "
<< " tests" << utils::attr_value() << tr.p_test_cases_passed
<< " skipped" << utils::attr_value() << tr.p_test_cases_skipped
<< " errors" << utils::attr_value() << tr.p_test_cases_aborted
<< " failures" << utils::attr_value() << tr.p_test_cases_failed
<< " id" << utils::attr_value() << m_id++
<< " name" << utils::attr_value() << tu_name_normalize(ts.p_name)
<< " time" << utils::attr_value() << (tr.p_duration_microseconds * 1E-6)
<< ">" << std::endl;
std::ostringstream o;
o << BOOST_VERSION/100000 << "." << BOOST_VERSION/100 % 1000 << "." << BOOST_VERSION % 100;
m_stream << "<property name=\"boost\" value" << utils::attr_value() << o.str() << std::endl;
m_stream << "</properties>" << std::endl;
if(m_display_build_info)
{
m_stream << "<properties>" << std::endl;
m_stream << "<property name=\"platform\" value" << utils::attr_value() << BOOST_PLATFORM << std::endl;
m_stream << "<property name=\"compiler\" value" << utils::attr_value() << BOOST_COMPILER << std::endl;
m_stream << "<property name=\"stl\" value" << utils::attr_value() << BOOST_STDLIB << std::endl;
std::ostringstream o;
o << BOOST_VERSION/100000 << "." << BOOST_VERSION/100 % 1000 << "." << BOOST_VERSION % 100;
m_stream << "<property name=\"boost\" value" << utils::attr_value() << o.str() << std::endl;
m_stream << "</properties>" << std::endl;
}
}
if( !tr.p_skipped ) {
// if we land here, then this is a chance that we are logging the fixture setup/teardown of a test-suite.
// the setup/teardown logging of a test-case is part of the test case.
// we do not care about the test-suite that were skipped (really??)
junit_log_formatter::map_trace_t::const_iterator it_find = m_map_test.find(ts.p_id);
if(it_find != m_map_test.end()) {
output_detailed_logs(it_find->second, ts, false, &tr);
}
}
return true; // indicates that the children should also be parsed
@@ -389,13 +457,13 @@ public:
virtual void test_suite_finish( test_suite const& ts )
{
if( m_ts.p_id != ts.p_id )
if( m_ts.p_id == ts.p_id ) {
write_testcase_system_out(runner_log, 0, false);
write_testcase_system_err(runner_log, 0, 0);
m_stream << "</testsuite>";
return;
write_testcase_system_out(runner_log, 0, false, 0);
write_testcase_system_err(runner_log, 0, 0);
m_stream << "</testsuite>";
}
}
private:

6
include/boost/test/impl/results_collector.ipp
View File

@@ -59,6 +59,12 @@ test_results::passed() const
!p_aborted;
}
bool
test_results::aborted() const
{
return p_aborted;
}
//____________________________________________________________________________//
int

109
include/boost/test/impl/test_framework_init_observer.ipp Normal file
View File

@@ -0,0 +1,109 @@
// (c) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
//
//! @file
//! An observer for monitoring the success/failure of the other observers
// ***************************************************************************
#ifndef BOOST_TEST_FRAMEWORK_INIT_OBSERVER_IPP_021105GER
#define BOOST_TEST_FRAMEWORK_INIT_OBSERVER_IPP_021105GER
// Boost.Test
#include <boost/test/test_framework_init_observer.hpp>
#include <boost/test/framework.hpp>
#include <boost/test/detail/suppress_warnings.hpp>
//____________________________________________________________________________//
namespace boost {
namespace unit_test {
//____________________________________________________________________________//
// ************************************************************************** //
// ************** framework_init_observer_t ************** //
// ************************************************************************** //
namespace {
struct test_init_observer_check {
bool has_failure;
void clear()
{
has_failure = false;
}
};
test_init_observer_check& s_tioc_impl() { static test_init_observer_check the_inst; return the_inst; }
} // local namespace
void
framework_init_observer_t::clear()
{
if(!framework::test_in_progress())
s_tioc_impl().clear();
}
//____________________________________________________________________________//
void
framework_init_observer_t::test_start( counter_t )
{
clear();
}
//____________________________________________________________________________//
void
framework_init_observer_t::assertion_result( unit_test::assertion_result ar )
{
test_init_observer_check& tr = s_tioc_impl();
switch( ar ) {
case AR_TRIGGERED: break;
case AR_PASSED: break;
case AR_FAILED: tr.has_failure = true; break;
default:
break;
}
}
//____________________________________________________________________________//
void
framework_init_observer_t::exception_caught( execution_exception const& )
{
test_init_observer_check& tr = s_tioc_impl();
tr.has_failure = true;
}
void
framework_init_observer_t::test_aborted()
{
s_tioc_impl().has_failure = true;
}
//____________________________________________________________________________//
bool
framework_init_observer_t::has_failed() const
{
return s_tioc_impl().has_failure;
}
//____________________________________________________________________________//
} // namespace unit_test
} // namespace boost
#include <boost/test/detail/enable_warnings.hpp>
#endif // BOOST_TEST_FRAMEWORK_INIT_OBSERVER_IPP_021105GER

43
include/boost/test/impl/test_tools.ipp
View File

@@ -310,8 +310,19 @@ report_assertion( assertion_result const& ar,
{
using namespace unit_test;
BOOST_TEST_I_ASSRT( framework::current_test_case_id() != INV_TEST_UNIT_ID,
std::runtime_error( "Can't use testing tools outside of test case implementation." ) );
if( !framework::test_in_progress() ) {
// in case no test is in progress, we do not throw anything:
// raising an exception here may result in raising an exception in a destructor of a global fixture
// which will abort the process
// We flag this as aborted instead
//BOOST_TEST_I_ASSRT( framework::current_test_case_id() != INV_TEST_UNIT_ID,
// std::runtime_error( "Can't use testing tools outside of test case implementation." ) );
framework::test_aborted();
return false;
}
if( !!ar )
tl = PASS;
@@ -369,10 +380,9 @@ report_assertion( assertion_result const& ar,
case REQUIRE:
framework::assertion_result( AR_FAILED );
framework::test_unit_aborted( framework::current_test_case() );
framework::test_unit_aborted( framework::current_test_unit() );
BOOST_TEST_I_THROW( execution_aborted() );
return false;
}
return true;
@@ -493,7 +503,7 @@ struct output_test_stream::Impl
char get_char()
{
char res;
char res = 0;
do {
m_pattern.get( res );
} while( m_text_or_binary && res == '\r' && !m_pattern.fail() && !m_pattern.eof() );
@@ -619,6 +629,7 @@ output_test_stream::match_pattern( bool flush_stream )
int offset = 0;
std::vector<char> last_elements;
for ( std::string::size_type i = 0; static_cast<int>(i + offset) < static_cast<int>(stream_string_repr.length()); ++i ) {
char c = m_pimpl->get_char();
if( last_elements.size() <= n_chars_presuffix ) {
@@ -698,7 +709,7 @@ output_test_stream::match_pattern( bool flush_stream )
if( last_elements_ordered[pattern_start_index + k] == sub_str_suffix[stream_start_index + k] )
nb_char_in_common ++;
else
break; // we take fully macthing substring only
break; // we take fully matching substring only
}
if( nb_char_in_common > max_nb_char_in_common ) {
@@ -709,20 +720,32 @@ output_test_stream::match_pattern( bool flush_stream )
}
}
// indicates with more precision the location of the mismatchs in ascii arts ...
// indicates with more precision the location of the mismatchs in "ascii arts" ...
result.message() << " ...\n... ";
for( std::string::size_type j = 0; j < sub_str_prefix.size(); j++) {
result.message() << ' ';
}
for( std::size_t k = 0; k < (std::max)(best_pattern_start_index, best_stream_start_index); k++ ) { // 1 is for the current char c
result.message() << '~'; // places the first tilde at the current char that mismatches
for( std::size_t k = 1; k < (std::max)(best_pattern_start_index, best_stream_start_index); k++ ) { // 1 is for the current char c
std::string s1(pretty_print_log(std::string(1, last_elements_ordered[(std::min)(k, best_pattern_start_index)])));
std::string s2(pretty_print_log(std::string(1, sub_str_suffix[(std::min)(k, best_stream_start_index)])));
for( int h = (std::max)(s1.size(), s2.size()); h > 0; h--)
result.message() << "~";
result.message() << "~";
}
if( m_pimpl->m_pattern.eof() ) {
result.message() << " (reference string shorter than current stream)";
}
result.message() << "\n";
// no need to continue if the EOF is reached
if( m_pimpl->m_pattern.eof() ) {
break;
}
// first char is a replicat of c, so we do not copy it.
for(std::string::size_type counter = 0; counter < last_elements_ordered.size() - 1 ; counter++)
last_elements[ (i + 1 + counter) % last_elements.size() ] = last_elements_ordered[counter + 1];

19
include/boost/test/impl/test_tree.ipp
View File

@@ -447,10 +447,29 @@ auto_test_unit_registrar::auto_test_unit_registrar( int )
// ************************************************************************** //
global_fixture::global_fixture()
{
framework::register_global_fixture( *this );
}
global_fixture::~global_fixture()
{
framework::deregister_global_fixture( *this );
}
// ************************************************************************** //
// ************** global_configuration ************** //
// ************************************************************************** //
global_configuration::global_configuration()
{
framework::register_observer( *this );
}
global_configuration::~global_configuration()
{
framework::deregister_observer( *this );
}
//____________________________________________________________________________//
} // namespace unit_test

2
include/boost/test/impl/unit_test_log.ipp
View File

@@ -171,6 +171,8 @@ unit_test_log_t::test_start( counter_t test_cases_amount )
if( runtime_config::get<bool>( runtime_config::btrt_build_info ) )
current_logger_data.m_log_formatter->log_build_info( current_logger_data.stream() );
//current_logger_data.stream().flush();
current_logger_data.m_entry_in_progress = false;
}
}

2
include/boost/test/impl/unit_test_monitor.ipp
View File

@@ -47,7 +47,7 @@ unit_test_monitor_t::execute_and_translate( boost::function<void ()> const& func
}
BOOST_TEST_I_CATCH( execution_exception, ex ) {
framework::exception_caught( ex );
framework::test_unit_aborted( framework::current_test_case() );
framework::test_unit_aborted( framework::current_test_unit() );
// translate execution_exception::error_code to error_level
switch( ex.code() ) {

1
include/boost/test/included/test_exec_monitor.hpp
View File

@@ -23,6 +23,7 @@
#include <boost/test/impl/progress_monitor.ipp>
#include <boost/test/impl/results_collector.ipp>
#include <boost/test/impl/results_reporter.ipp>
#include <boost/test/impl/test_framework_init_observer.ipp>
#include <boost/test/impl/test_main.ipp>
#include <boost/test/impl/test_tools.ipp>
#include <boost/test/impl/test_tree.ipp>

3
include/boost/test/included/unit_test.hpp
View File

@@ -5,6 +5,7 @@
// See http://www.boost.org/libs/test for the library home page.
//
//
//!@file
//!@brief Included (vs. linked) version of Unit Test Framework
// ***************************************************************************
@@ -24,6 +25,7 @@
#include <boost/test/impl/progress_monitor.ipp>
#include <boost/test/impl/results_collector.ipp>
#include <boost/test/impl/results_reporter.ipp>
#include <boost/test/impl/test_framework_init_observer.ipp>
#include <boost/test/impl/test_tools.ipp>
#include <boost/test/impl/test_tree.ipp>
#include <boost/test/impl/unit_test_log.ipp>
@@ -31,7 +33,6 @@
#include <boost/test/impl/unit_test_monitor.ipp>
#include <boost/test/impl/unit_test_parameters.ipp>
#include <boost/test/impl/xml_log_formatter.ipp>
#include <boost/test/impl/junit_log_formatter.ipp>
#include <boost/test/impl/xml_report_formatter.ipp>
#include <boost/test/unit_test.hpp>

2
include/boost/test/progress_monitor.hpp
View File

@@ -41,7 +41,7 @@ public:
virtual void test_unit_finish( test_unit const&, unsigned long );
virtual void test_unit_skipped( test_unit const&, const_string );
virtual int priority() { return 3; }
virtual int priority() { return 4; }
/// @}
/// @name Configuration

5
include/boost/test/results_collector.hpp
View File

@@ -79,6 +79,9 @@ public:
/// Returns true if test unit passed
bool passed() const;
/// Returns true if the test unit was aborted (hard failure)
bool aborted() const;
/// Produces result code for the test unit execution
///
/// This methhod return one of the result codes defined in @c boost/cstdlib.hpp
@@ -119,7 +122,7 @@ public:
virtual void assertion_result( unit_test::assertion_result );
virtual void exception_caught( execution_exception const& );
virtual int priority() { return 2; }
virtual int priority() { return 3; }
/// Results access per test unit
///

63
include/boost/test/test_framework_init_observer.hpp Normal file
View File

@@ -0,0 +1,63 @@
// (c) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
//
/// @file
/// @brief Defines an observer that monitors the init of the unit test framework
// ***************************************************************************
#ifndef BOOST_TEST_FRAMEWORK_INIT_OBSERVER_HPP_071894GER
#define BOOST_TEST_FRAMEWORK_INIT_OBSERVER_HPP_071894GER
// Boost.Test
#include <boost/test/tree/observer.hpp>
#include <boost/test/detail/global_typedef.hpp>
#include <boost/test/detail/fwd_decl.hpp>
#include <boost/test/utils/trivial_singleton.hpp>
#include <boost/test/detail/suppress_warnings.hpp>
//____________________________________________________________________________//
namespace boost {
namespace unit_test {
// ************************************************************************** //
/// @brief Monitors the init of the framework
///
/// This class collects the state of the init/termination of the unit test framework.
///
/// @see boost::unit_test::test_observer
class BOOST_TEST_DECL framework_init_observer_t : public test_observer, public singleton<framework_init_observer_t> {
public:
virtual void test_start( counter_t );
virtual void assertion_result( unit_test::assertion_result );
virtual void exception_caught( execution_exception const& );
virtual void test_aborted();
virtual int priority() { return 0; }
void clear();
/// Indicates if a failure has been recorded so far
bool has_failed( ) const;
private:
BOOST_TEST_SINGLETON_CONS( framework_init_observer_t )
};
BOOST_TEST_SINGLETON_INST( framework_init_observer )
} // namespace unit_test
} // namespace boost
#include <boost/test/detail/enable_warnings.hpp>
#endif // BOOST_TEST_FRAMEWORK_INIT_OBSERVER_HPP_071894GER

93
include/boost/test/tree/fixture.hpp
View File

@@ -5,11 +5,8 @@
// See http://www.boost.org/libs/test for the library home page.
//
// File : $RCSfile$
//
// Version : $Revision: 74640 $
//
// Description : defines fixture interface and object makers
/// @file
/// Defines fixture interface and object makers
// ***************************************************************************
#ifndef BOOST_TEST_TREE_FIXTURE_HPP_100311GER
@@ -22,6 +19,7 @@
#include <boost/shared_ptr.hpp>
#include <boost/scoped_ptr.hpp>
#include <boost/function/function0.hpp>
#include <boost/utility/declval.hpp>
#include <boost/test/detail/suppress_warnings.hpp>
@@ -45,6 +43,83 @@ public:
typedef shared_ptr<test_unit_fixture> test_unit_fixture_ptr;
// ************************************************************************** //
// ************** fixture helper functions ************** //
// ************************************************************************** //
namespace impl_fixture {
#if defined(BOOST_NO_CXX11_DECLTYPE) || defined(BOOST_NO_CXX11_TRAILING_RESULT_TYPES)
template<typename U, void (U::*)()> struct fixture_detect {};
template<typename T>
struct has_setup {
private:
template<typename U> static char Test(fixture_detect<U, &U::setup>*);
template<typename U> static int Test(...);
public:
static const bool value = sizeof(Test<T>(0)) == sizeof(char);
};
template<typename T>
struct has_teardown {
private:
template<typename U> static char Test(fixture_detect<U, &U::teardown>*);
template<typename U> static int Test(...);
public:
static const bool value = sizeof(Test<T>(0)) == sizeof(char);
};
#else
template<typename U> struct fixture_detect { typedef char type; };
template<typename T>
struct has_setup {
private:
template<typename U> static auto Test(U*) -> typename fixture_detect<decltype(boost::declval<U>().setup())>::type;
template<typename U> static int Test(...);
public:
static const bool value = sizeof(Test<T>(0)) == sizeof(char);
};
template<typename T>
struct has_teardown {
private:
template<typename U> static auto Test(U*) -> typename fixture_detect<decltype(boost::declval<U>().teardown())>::type;
template<typename U> static int Test(...);
public:
static const bool value = sizeof(Test<T>(0)) == sizeof(char);
};
#endif
template <bool has_setup = false>
struct call_setup { template <class U> void operator()(U& ) { } };
template <>
struct call_setup<true> { template <class U> void operator()(U& u) { u.setup(); } };
template <bool has_teardown = false>
struct call_teardown { template <class U> void operator()(U& ) { } };
template <>
struct call_teardown<true> { template <class U> void operator()(U& u) { u.teardown(); } };
}
//! Calls the fixture "setup" if detected by the compiler, otherwise does nothing.
template <class U>
void setup_conditional(U& u) {
return impl_fixture::call_setup<impl_fixture::has_setup<U>::value>()(u);
}
//! Calls the fixture "teardown" if detected by the compiler, otherwise does nothing.
template <class U>
void teardown_conditional(U& u) {
return impl_fixture::call_teardown<impl_fixture::has_teardown<U>::value>()(u);
}
// ************************************************************************** //
// ************** class_based_fixture ************** //
// ************************************************************************** //
@@ -57,8 +132,8 @@ public:
private:
// Fixture interface
virtual void setup() { m_inst.reset( new F( m_arg ) ); }
virtual void teardown() { m_inst.reset(); }
virtual void setup() { m_inst.reset( new F( m_arg ) ); setup_conditional(*m_inst); }
virtual void teardown() { teardown_conditional(*m_inst); m_inst.reset(); }
// Data members
scoped_ptr<F> m_inst;
@@ -75,8 +150,8 @@ public:
private:
// Fixture interface
virtual void setup() { m_inst.reset( new F ); }
virtual void teardown() { m_inst.reset(); }
virtual void setup() { m_inst.reset( new F ); setup_conditional(*m_inst); }
virtual void teardown() { teardown_conditional(*m_inst); m_inst.reset(); }
// Data members
scoped_ptr<F> m_inst;

70
include/boost/test/tree/global_fixture.hpp
View File

@@ -17,6 +17,7 @@
#include <boost/test/detail/global_typedef.hpp>
#include <boost/test/tree/observer.hpp>
#include <boost/test/tree/fixture.hpp>
#include <boost/test/detail/suppress_warnings.hpp>
@@ -26,14 +27,37 @@
namespace boost {
namespace unit_test {
// ************************************************************************** //
// ************** global_configuration ************** //
// ************************************************************************** //
class BOOST_TEST_DECL global_configuration : public test_observer {
public:
// Constructor
global_configuration();
// Dtor
virtual ~global_configuration();
// Happens after the framework global observer init has been done
virtual int priority() { return 1; }
};
// ************************************************************************** //
// ************** global_fixture ************** //
// ************************************************************************** //
class BOOST_TEST_DECL global_fixture : public test_observer {
class BOOST_TEST_DECL global_fixture : public test_unit_fixture {
public:
// Constructor
global_fixture();
// Dtor
virtual ~global_fixture();
};
//____________________________________________________________________________//
@@ -41,14 +65,48 @@ public:
namespace ut_detail {
template<typename F>
struct global_fixture_impl : public global_fixture {
struct global_configuration_impl : public global_configuration {
// Constructor
global_fixture_impl() : m_fixture( 0 ) {}
global_configuration_impl() : m_configuration_observer( 0 ) {
}
// test observer interface
virtual void test_start( counter_t ) { m_fixture = new F; }
virtual void test_finish() { delete m_fixture; m_fixture = 0; }
virtual void test_aborted() { delete m_fixture; m_fixture = 0; }
virtual void test_start( counter_t ) {
m_configuration_observer = new F;
}
// test observer interface
virtual void test_finish() {
if(m_configuration_observer) {
delete m_configuration_observer;
m_configuration_observer = 0;
}
}
private:
// Data members
F* m_configuration_observer;
};
template<typename F>
struct global_fixture_impl : public global_fixture {
// Constructor
global_fixture_impl() : m_fixture( 0 ) {
}
// test fixture interface
virtual void setup() {
m_fixture = new F;
setup_conditional(*m_fixture);
}
// test fixture interface
virtual void teardown() {
if(m_fixture) {
teardown_conditional(*m_fixture);
}
delete m_fixture;
m_fixture = 0;
}
private:
// Data members

6
include/boost/test/tree/observer.hpp
View File

@@ -34,7 +34,7 @@ namespace unit_test {
/// Boost.Test framework on the current execution state.
///
/// Several observers can be running at the same time, and it is not unusual to
/// have interactions among them. The test_observer#priority member function allows the specification
/// have interactions among them. The @ref test_observer::priority member function allows the specification
/// of a particular order among them (lowest priority executed first, except specified otherwise).
///
class BOOST_TEST_DECL test_observer {
@@ -44,10 +44,8 @@ public:
//!
//! @param[in] number_of_test_cases indicates the number of test cases. Only active
//! test cases are taken into account.
//!
virtual void test_start( counter_t /* number_of_test_cases */ ) {}
//! Called after the framework ends executing the test cases
//!
//! @note The call is made with a reversed priority order.
@@ -98,6 +96,8 @@ public:
//! additional data about the exception.
virtual void exception_caught( execution_exception const& ) {}
//! The priority indicates the order at which this observer is initialized
//! and tore down in the UTF framework. The order is lowest to highest priority.
virtual int priority() { return 0; }
protected:

2
include/boost/test/unit_test_log.hpp
View File

@@ -123,7 +123,7 @@ public:
virtual void exception_caught( execution_exception const& ex );
virtual int priority() { return 1; }
virtual int priority() { return 2; }
// log configuration methods
//! Sets the stream for all loggers

29
include/boost/test/unit_test_suite.hpp
View File

@@ -143,11 +143,15 @@ struct test_name : public F { void test_method(); }; \
\
static void BOOST_AUTO_TC_INVOKER( test_name )() \
{ \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture entry."); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture ctor"); \
test_name t; \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" entry."); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture setup"); \
boost::unit_test::setup_conditional(t); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" test entry"); \
t.test_method(); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" exit."); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture teardown"); \
boost::unit_test::teardown_conditional(t); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture dtor"); \
} \
\
struct BOOST_AUTO_TC_UNIQUE_ID( test_name ) {}; \
@@ -230,10 +234,11 @@ struct BOOST_AUTO_TC_INVOKER( test_name ) { \
static void run( boost::type<TestType>* = 0 ) \
{ \
BOOST_TEST_CHECKPOINT('"' << #test_name <<"\" fixture entry."); \
test_name<TestType> t; \
test_name<TestType> t; boost::unit_test::setup_conditional(t); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" entry."); \
t.test_method(); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" exit."); \
boost::unit_test::teardown_conditional(t); \
} \
}; \
\
@@ -290,6 +295,22 @@ void BOOST_JOIN( name, _impl )( boost::type<type_name>* ) \
// ************************************************************************** //
#define BOOST_GLOBAL_FIXTURE( F ) \
static boost::unit_test::ut_detail::global_configuration_impl<F> BOOST_JOIN( gf_, F ) \
/**/
// ************************************************************************** //
// ************** BOOST_TEST_GLOBAL_CONFIGURATION ************** //
// ************************************************************************** //
#define BOOST_TEST_GLOBAL_CONFIGURATION( F ) \
static boost::unit_test::ut_detail::global_configuration_impl<F> BOOST_JOIN( gf_, F ) \
/**/
// ************************************************************************** //
// ************** BOOST_TEST_GLOBAL_FIXTURE ************** //
// ************************************************************************** //
#define BOOST_TEST_GLOBAL_FIXTURE( F ) \
static boost::unit_test::ut_detail::global_fixture_impl<F> BOOST_JOIN( gf_, F ) \
/**/

15
src/test_framework_init_observer.cpp Normal file
View File

@@ -0,0 +1,15 @@
// (C) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
//
//! @file
//! Forwarding source
// **************************************************************************
#define BOOST_TEST_SOURCE
#include <boost/test/impl/test_framework_init_observer.ipp>
// EOF

3
test/Jamfile.v2
View File

@@ -136,6 +136,7 @@ test-suite "framework-ts"
[ boost.test-self-test run : framework-ts : log-formatter-test : : baseline-outputs/log-formatter-test.pattern ]
[ boost.test-self-test run : framework-ts : run-by-name-or-label-test ]
[ boost.test-self-test run : framework-ts : version-uses-module-name : included ]
[ boost.test-self-test run : framework-ts : test-macro-global-fixture : : baseline-outputs/global-fixtures-test.pattern ]
;
#_________________________________________________________________________________________________#
@@ -156,6 +157,8 @@ test-suite "writing-test-ts"
[ boost.test-self-test run : writing-test-ts : test-dataset-over-tuples : : : : : : $(requirements_datasets) ]
[ boost.test-self-test run : writing-test-ts : nullptr-support-test : : : : : : [ requires cxx11_nullptr ] ]
[ boost.test-self-test run : writing-test-ts : user-defined-types-logging-customization-points ]
[ boost.test-self-test run : writing-test-ts : test-fixture-detect-setup-teardown ]
[ boost.test-self-test run : writing-test-ts : test-fixture-detect-setup-teardown-cpp11 : : : : : : [ requires cxx11_decltype cxx11_trailing_result_types ] ]
;
#_________________________________________________________________________________________________#

5831
test/baseline-outputs/global-fixtures-test.pattern Normal file
View File

File diff suppressed because it is too large Load Diff

46
test/baseline-outputs/log-formatter-test.pattern
View File

@@ -7,14 +7,14 @@ xxx/log-formatter-test.cpp:210: Leaving test case "good_foo"
xxx/log-formatter-test.cpp:209: Leaving test suite "1 test cases inside"
* 2-format *******************************************************************
<TestLog><TestSuite name="1 test cases inside" file="xxx/log-formatter-test.cpp" line="209"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="210"><Message file="boost.test framework" line="206"><![CDATA[Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
<TestLog><TestSuite name="1 test cases inside" file="xxx/log-formatter-test.cpp" line="209"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="210"><Message file="boost.test framework" line="212"><![CDATA[Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
* 3-format *******************************************************************
<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="1" skipped="0" errors="0" failures="0" id="0" name="1_test_cases_inside" time="0.1234">
<testcase assertions="0" name="good_foo" time="0.1234">
<system-out><![CDATA[MESSAGE:
- file : boost.test framework
- line : 206
- line : 212
- message: Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions
]]></system-out>
@@ -82,7 +82,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: 1 bad test case inside/bad_foo
- file: log-formatter-test.cpp
- line: 213
]]></system-err>
@@ -116,7 +115,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: 1 bad test case inside/bad_foo
- file: log-formatter-test.cpp
- line: 213
]]></system-err>
@@ -132,7 +130,7 @@ xxx/log-formatter-test.cpp:216: Leaving test case "almost_good_foo"
xxx/log-formatter-test.cpp:215: Leaving test suite "1 almost good test case inside"
* 2-format *******************************************************************
<TestLog><TestSuite name="1 almost good test case inside" file="xxx/log-formatter-test.cpp" line="215"><TestCase name="almost_good_foo" file="xxx/log-formatter-test.cpp" line="216"><Warning file="xxx/log-formatter-test.cpp" line="43"><![CDATA[condition 2>3 is not satisfied [2 <= 3]]]></Warning><Message file="boost.test framework" line="206"><![CDATA[Test case 1 almost good test case inside/almost_good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
<TestLog><TestSuite name="1 almost good test case inside" file="xxx/log-formatter-test.cpp" line="215"><TestCase name="almost_good_foo" file="xxx/log-formatter-test.cpp" line="216"><Warning file="xxx/log-formatter-test.cpp" line="43"><![CDATA[condition 2>3 is not satisfied [2 <= 3]]]></Warning><Message file="boost.test framework" line="212"><![CDATA[Test case 1 almost good test case inside/almost_good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
* 3-format *******************************************************************
<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="0" skipped="0" errors="0" failures="0" id="0" name="1_almost_good_test_case_inside" time="0.1234">
@@ -144,7 +142,7 @@ xxx/log-formatter-test.cpp:215: Leaving test suite "1 almost good test case insi
MESSAGE:
- file : boost.test framework
- line : 206
- line : 212
- message: Test case 1 almost good test case inside/almost_good_foo did not check any assertions
]]></system-out>
@@ -175,14 +173,14 @@ xxx/log-formatter-test.cpp:220: Leaving test case "bad_foo"
xxx/log-formatter-test.cpp:218: Leaving test suite "2 test cases inside"
* 2-format *******************************************************************
<TestLog><TestSuite name="2 test cases inside" file="xxx/log-formatter-test.cpp" line="218"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="219"><Message file="boost.test framework" line="206"><![CDATA[Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="220"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
<TestLog><TestSuite name="2 test cases inside" file="xxx/log-formatter-test.cpp" line="218"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="219"><Message file="boost.test framework" line="212"><![CDATA[Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="220"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite></TestLog>
* 3-format *******************************************************************
<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="1" skipped="0" errors="0" failures="1" id="0" name="2_test_cases_inside" time="0.1234">
<testcase assertions="0" name="good_foo" time="0.1234">
<system-out><![CDATA[MESSAGE:
- file : boost.test framework
- line : 206
- line : 212
- message: Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions
]]></system-out>
@@ -223,7 +221,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/2 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 220
]]></system-err>
@@ -259,7 +256,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/2 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 220
]]></system-err>
@@ -327,7 +323,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/3 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 223
]]></system-err>
@@ -344,7 +339,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/3 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 224
]]></system-err>
@@ -384,7 +378,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/3 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 223
]]></system-err>
@@ -401,7 +394,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/3 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 224
]]></system-err>
@@ -494,7 +486,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 231
]]></system-err>
@@ -511,7 +502,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 232
]]></system-err>
@@ -547,7 +537,6 @@ CONTEXT:
- 'exception context should be shown'
]]></error><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_exception
- file: log-formatter-test.cpp
- line: 233
]]></system-err>
@@ -588,7 +577,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 234
]]></system-err>
@@ -622,7 +610,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 231
]]></system-err>
@@ -639,7 +626,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 232
]]></system-err>
@@ -675,7 +661,6 @@ CONTEXT:
- 'exception context should be shown'
]]></error><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_exception
- file: log-formatter-test.cpp
- line: 233
]]></system-err>
@@ -705,7 +690,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 234
]]></system-err>
@@ -785,14 +769,14 @@ xxx/log-formatter-test.cpp:222: Test suite "Fake Test Suite Hierarchy/3 test cas
xxx/log-formatter-test.cpp:236: Leaving test suite "Fake Test Suite Hierarchy"
* 2-format *******************************************************************
<TestLog><TestSuite name="Fake Test Suite Hierarchy" file="xxx/log-formatter-test.cpp" line="236"><TestSuite name="1 test cases inside" file="xxx/log-formatter-test.cpp" line="209"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="210"><Message file="boost.test framework" line="206"><![CDATA[Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="255"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="2 test cases inside" file="xxx/log-formatter-test.cpp" line="218"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="219"><Message file="boost.test framework" line="206"><![CDATA[Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="220"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="4 test cases inside" file="xxx/log-formatter-test.cpp" line="230"><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="231"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="very_bad_foo" file="xxx/log-formatter-test.cpp" line="232"><FatalError file="xxx/log-formatter-test.cpp" line="68"><![CDATA[very_bad_foo is fatal]]><Context><Frame><![CDATA[some context]]></Frame></Context></FatalError><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="very_bad_exception" file="xxx/log-formatter-test.cpp" line="233"><Error file="xxx/log-formatter-test.cpp" line="77"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Exception file="unknown location" line="0"><![CDATA[unknown type]]><LastCheckpoint file="xxx/log-formatter-test.cpp" line="77"><![CDATA[]]></LastCheckpoint><Context><Frame><![CDATA[exception context should be shown]]></Frame></Context></Exception><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="234"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="3 test cases inside" skipped="yes" reason="dependency test suite &quot;Fake Test Suite Hierarchy/1 test cases inside&quot; has failed"/></TestSuite></TestLog>
<TestLog><TestSuite name="Fake Test Suite Hierarchy" file="xxx/log-formatter-test.cpp" line="236"><TestSuite name="1 test cases inside" file="xxx/log-formatter-test.cpp" line="209"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="210"><Message file="boost.test framework" line="212"><![CDATA[Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="255"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="2 test cases inside" file="xxx/log-formatter-test.cpp" line="218"><TestCase name="good_foo" file="xxx/log-formatter-test.cpp" line="219"><Message file="boost.test framework" line="212"><![CDATA[Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions]]></Message><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="220"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="4 test cases inside" file="xxx/log-formatter-test.cpp" line="230"><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="231"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="very_bad_foo" file="xxx/log-formatter-test.cpp" line="232"><FatalError file="xxx/log-formatter-test.cpp" line="68"><![CDATA[very_bad_foo is fatal]]><Context><Frame><![CDATA[some context]]></Frame></Context></FatalError><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="very_bad_exception" file="xxx/log-formatter-test.cpp" line="233"><Error file="xxx/log-formatter-test.cpp" line="77"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Exception file="unknown location" line="0"><![CDATA[unknown type]]><LastCheckpoint file="xxx/log-formatter-test.cpp" line="77"><![CDATA[]]></LastCheckpoint><Context><Frame><![CDATA[exception context should be shown]]></Frame></Context></Exception><TestingTime>ZZZ</TestingTime></TestCase><TestCase name="bad_foo" file="xxx/log-formatter-test.cpp" line="234"><Error file="xxx/log-formatter-test.cpp" line="47"><![CDATA[]]></Error><Message file="xxx/log-formatter-test.cpp" line="49"><![CDATA[this is a message]]></Message><Info file="xxx/log-formatter-test.cpp" line="50"><![CDATA[check true has passed]]></Info><Error file="xxx/log-formatter-test.cpp" line="54"><![CDATA[with some message]]><Context><Frame><![CDATA[Context value=something]]></Frame><Frame><![CDATA[Context value2=something different]]></Frame></Context></Error><Error file="xxx/log-formatter-test.cpp" line="56"><![CDATA[non sense]]></Error><TestingTime>ZZZ</TestingTime></TestCase></TestSuite><TestSuite name="3 test cases inside" skipped="yes" reason="dependency test suite &quot;Fake Test Suite Hierarchy/1 test cases inside&quot; has failed"/></TestSuite></TestLog>
* 3-format *******************************************************************
<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="2" skipped="3" errors="2" failures="6" id="0" name="Fake_Test_Suite_Hierarchy" time="0.1234">
<testcase assertions="0" classname="1_test_cases_inside" name="good_foo" time="0.1234">
<system-out><![CDATA[MESSAGE:
- file : boost.test framework
- line : 206
- line : 212
- message: Test case Fake Test Suite Hierarchy/1 test cases inside/good_foo did not check any assertions
]]></system-out>
@@ -833,7 +817,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/1 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 255
]]></system-err>
@@ -841,7 +824,7 @@ INFO:
<testcase assertions="0" classname="2_test_cases_inside" name="good_foo" time="0.1234">
<system-out><![CDATA[MESSAGE:
- file : boost.test framework
- line : 206
- line : 212
- message: Test case Fake Test Suite Hierarchy/2 test cases inside/good_foo did not check any assertions
]]></system-out>
@@ -882,7 +865,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/2 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 220
]]></system-err>
@@ -944,7 +926,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 231
]]></system-err>
@@ -961,7 +942,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 232
]]></system-err>
@@ -997,7 +977,6 @@ CONTEXT:
- 'exception context should be shown'
]]></error><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_exception
- file: log-formatter-test.cpp
- line: 233
]]></system-err>
@@ -1038,7 +1017,6 @@ INFO:
]]></system-out>
<system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 234
]]></system-err>
@@ -1074,7 +1052,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/1 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 255
]]></system-err>
@@ -1106,7 +1083,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/2 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 220
]]></system-err>
@@ -1157,7 +1133,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 231
]]></system-err>
@@ -1174,7 +1149,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_foo
- file: log-formatter-test.cpp
- line: 232
]]></system-err>
@@ -1210,7 +1184,6 @@ CONTEXT:
- 'exception context should be shown'
]]></error><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/very_bad_exception
- file: log-formatter-test.cpp
- line: 233
]]></system-err>
@@ -1240,7 +1213,6 @@ ASSERTION FAILURE:
]]></failure><system-err><![CDATA[Failures detected in:
- test case: Fake Test Suite Hierarchy/4 test cases inside/bad_foo
- file: log-formatter-test.cpp
- line: 234
]]></system-err>

6
test/framework-ts/log-formatter-test.cpp
View File

@@ -11,7 +11,7 @@
// Boost.Test
#define BOOST_TEST_MAIN
#include <boost/test/included/unit_test.hpp>
#include <boost/test/unit_test.hpp>
#include <boost/test/unit_test_log.hpp>
#include <boost/test/tools/output_test_stream.hpp>
#include <boost/test/unit_test_suite.hpp>
@@ -113,7 +113,7 @@ void check( output_test_stream& output, test_suite* ts )
check( output, OF_CLF, ts->p_id );
check( output, OF_XML, ts->p_id );
check( output, OF_JUNIT, ts->p_id, log_successful_tests );
check( output, OF_JUNIT, ts->p_id, log_cpp_exception_errors ); // should branch to the log log_all_errors
check( output, OF_JUNIT, ts->p_id, log_cpp_exception_errors ); // should branch to the log log_all_errors
}
//____________________________________________________________________________//
@@ -186,7 +186,7 @@ public:
"condition 2>3 is not satisfied\n",
"condition 2>3 is not satisfied]",
};
static const std::string to_replace[] = {"time=\"0.1234\"",
get_basename() + ":*:" ,
"unknown location:*:",

417
test/framework-ts/test-macro-global-fixture.cpp Normal file
View File

@@ -0,0 +1,417 @@
// (C) Copyright Raffi Enficiaud 2016.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
// checks issue https://svn.boost.org/trac/boost/ticket/5563
#define BOOST_TEST_MODULE test_macro_in_global_fixture
#include <boost/test/unit_test.hpp>
#include <boost/test/unit_test_log.hpp>
#include <boost/test/results_collector.hpp>
#include <boost/test/tools/output_test_stream.hpp>
#include <boost/test/unit_test_suite.hpp>
#include <boost/test/framework.hpp>
#include <boost/test/unit_test_parameters.hpp>
#include <boost/test/utils/nullstream.hpp>
typedef boost::onullstream onullstream_type;
#include <boost/test/utils/algorithm.hpp>
// BOOST
#include <boost/lexical_cast.hpp>
// STL
#include <iostream>
#include <ios>
using boost::test_tools::output_test_stream;
using namespace boost::unit_test;
namespace utf = boost::unit_test;
template < void (*function_to_call)() >
struct GlobalFixtureWithCtor {
GlobalFixtureWithCtor() {
BOOST_TEST_MESSAGE("GlobalFixtureWithCtor: ctor");
(*function_to_call)();
// having a message is ok
// although it should break existing logger consistency
}
~GlobalFixtureWithCtor() {
BOOST_TEST_MESSAGE("GlobalFixtureWithCtor: dtor");
}
};
template < void (*function_to_call)() >
struct GlobalFixtureWithSetup {
GlobalFixtureWithSetup() {
BOOST_TEST_MESSAGE("GlobalFixtureWithSetup ctor");
}
virtual ~GlobalFixtureWithSetup() {
BOOST_TEST_MESSAGE("GlobalFixtureWithSetup dtor");
}
virtual void setup() {
BOOST_TEST_MESSAGE("GlobalFixtureWithSetup::setup-calling function");
(*function_to_call)();
BOOST_TEST_MESSAGE("GlobalFixtureWithSetup::setup-calling function done");
}
};
template < void (*function_to_call)() >
struct GlobalFixtureWithTeardown {
GlobalFixtureWithTeardown() {
BOOST_TEST_MESSAGE("GlobalFixtureWithTeardown ctor");
}
virtual ~GlobalFixtureWithTeardown() {
BOOST_TEST_MESSAGE("GlobalFixtureWithTeardown dtor");
}
virtual void teardown() {
BOOST_TEST_MESSAGE("GlobalFixtureWithTeardown::teardown-calling function");
(*function_to_call)();
BOOST_TEST_MESSAGE("GlobalFixtureWithTeardown::teardown-calling function done");
}
};
template <class global_fixture_t >
void check_global_fixture(
output_test_stream& output,
output_format log_format,
test_unit_id id,
bool bt_module_failed = false,
bool has_setup_error = false,
log_level ll = log_successful_tests )
{
boost::unit_test::unit_test_log.set_format(log_format);
boost::unit_test::unit_test_log.set_stream(output);
boost::unit_test::unit_test_log.set_threshold_level(ll);
// output before fixture registration
output << "* " << log_format << "-format *******************************************************************";
output << std::endl;
// register this as a global fixture
boost::unit_test::ut_detail::global_fixture_impl<global_fixture_t> fixture_stack_element;
framework::finalize_setup_phase( id );
bool setup_error_caught = false;
try {
framework::run( id, false ); // do not continue the test tree to have the test_log_start/end
}
catch (framework::setup_error&) {
BOOST_TEST_MESSAGE("Framework setup_error caught");
setup_error_caught = true;
}
output << std::endl;
// we do not want the result of the comparison go to the "output" stream
boost::unit_test::unit_test_log.set_format(OF_CLF);
boost::unit_test::unit_test_log.set_stream(std::cout);
BOOST_TEST( setup_error_caught == has_setup_error );
BOOST_TEST( bt_module_failed == (( results_collector.results( id ).result_code() != 0 ) || setup_error_caught ));
BOOST_TEST( output.match_pattern(true) ); // flushes the stream at the end of the comparison.
}
template <class global_fixture_t>
void check_global_fixture(
output_test_stream& output,
test_suite* ts,
bool bt_module_failed = false,
bool has_setup_error = false)
{
ts->p_default_status.value = test_unit::RS_ENABLED;
check_global_fixture<global_fixture_t>( output, OF_CLF, ts->p_id, bt_module_failed, has_setup_error );
check_global_fixture<global_fixture_t>( output, OF_XML, ts->p_id, bt_module_failed, has_setup_error );
check_global_fixture<global_fixture_t>( output, OF_JUNIT, ts->p_id, bt_module_failed, has_setup_error, log_successful_tests );
check_global_fixture<global_fixture_t>( output, OF_JUNIT, ts->p_id, bt_module_failed, has_setup_error, log_cpp_exception_errors ); // should branch to the log log_all_errors
}
struct guard {
~guard()
{
boost::unit_test::unit_test_log.set_format( runtime_config::get<output_format>( runtime_config::btrt_log_format ) );
boost::unit_test::unit_test_log.set_stream( std::cout );
}
};
// this one should generate a message as it does not execute any assertion
void good_foo() {}
void almost_good_foo()
{
BOOST_TEST_WARN( 2>3 );
}
void bad_foo() {
BOOST_ERROR( "" );
BOOST_TEST_MESSAGE("this is a message");
BOOST_CHECK(true);
BOOST_TEST_INFO("Context value=something");
BOOST_TEST_INFO("Context value2=something different");
BOOST_ERROR( "with some message" );
BOOST_CHECK_MESSAGE( 1 == 2.3, "non sense" );
}
struct log_guard {
~log_guard()
{
unit_test_log.set_stream( std::cout );
}
};
void very_bad_foo() {
BOOST_TEST_CONTEXT("some context") {
BOOST_FAIL( "very_bad_foo is fatal" );
}
}
struct local_exception {};
void very_bad_exception() {
BOOST_TEST_INFO("Context value=something");
BOOST_TEST_INFO("Context value2=something different");
BOOST_ERROR( "with some message" );
BOOST_TEST_INFO("exception context should be shown");
throw local_exception();
}
// to factorize out with the logger test
class output_test_stream_for_loggers : public output_test_stream {
public:
explicit output_test_stream_for_loggers(
boost::unit_test::const_string pattern_file_name = boost::unit_test::const_string(),
bool match_or_save = true,
bool text_or_binary = true )
: output_test_stream(pattern_file_name, match_or_save, text_or_binary)
{}
static std::string normalize_path(const std::string &str) {
const std::string to_look_for[] = {"\\"};
const std::string to_replace[] = {"/"};
return utils::replace_all_occurrences_of(
str,
to_look_for, to_look_for + sizeof(to_look_for)/sizeof(to_look_for[0]),
to_replace, to_replace + sizeof(to_replace)/sizeof(to_replace[0])
);
}
static std::string get_basename() {
static std::string basename;
if(basename.empty()) {
basename = normalize_path(__FILE__);
std::string::size_type basename_pos = basename.rfind('/');
if(basename_pos != std::string::npos) {
basename = basename.substr(basename_pos+1);
}
}
return basename;
}
virtual std::string get_stream_string_representation() const {
std::string current_string = output_test_stream::get_stream_string_representation();
std::string pathname_fixes;
{
static const std::string to_look_for[] = {normalize_path(__FILE__)};
static const std::string to_replace[] = {"xxx/" + get_basename() };
pathname_fixes = utils::replace_all_occurrences_of(
current_string,
to_look_for, to_look_for + sizeof(to_look_for)/sizeof(to_look_for[0]),
to_replace, to_replace + sizeof(to_replace)/sizeof(to_replace[0])
);
}
std::string other_vars_fixes;
{
static const std::string to_look_for[] = {"time=\"*\"",
get_basename() + "(*):",
"unknown location(*):",
"; testing time: *us\n", // removing this is far more easier than adding a testing time
"; testing time: *ms\n",
"<TestingTime>*</TestingTime>",
"condition 2>3 is not satisfied\n",
"condition 2>3 is not satisfied]",
};
static const std::string to_replace[] = {"time=\"0.1234\"",
get_basename() + ":*:" ,
"unknown location:*:",
"\n",
"\n",
"<TestingTime>ZZZ</TestingTime>",
"condition 2>3 is not satisfied [2 <= 3]\n",
"condition 2>3 is not satisfied [2 <= 3]]",
};
other_vars_fixes = utils::replace_all_occurrences_with_wildcards(
pathname_fixes,
to_look_for, to_look_for + sizeof(to_look_for)/sizeof(to_look_for[0]),
to_replace, to_replace + sizeof(to_replace)/sizeof(to_replace[0])
);
}
return other_vars_fixes;
}
};
BOOST_AUTO_TEST_CASE( some_test )
{
guard G;
ut_detail::ignore_unused_variable_warning( G );
test_suite* ts_1 = BOOST_TEST_SUITE( "1 test cases inside" );
ts_1->add( BOOST_TEST_CASE( good_foo ) );
test_suite* ts_1b = BOOST_TEST_SUITE( "1 bad test case inside" );
ts_1b->add( BOOST_TEST_CASE( bad_foo ), 1 );
test_suite* ts_main = BOOST_TEST_SUITE( "Fake Test Suite Hierarchy" );
ts_main->add( BOOST_TEST_CASE( bad_foo ) );
ts_main->add( BOOST_TEST_CASE( very_bad_foo ) );
ts_main->add( BOOST_TEST_CASE( very_bad_exception ) );
ts_main->add( ts_1 );
ts_main->add( ts_1b );
// we need another tree
test_suite* ts2_0 = BOOST_TEST_SUITE( "0 test cases inside" );
test_suite* ts2_1 = BOOST_TEST_SUITE( "1 test cases inside" );
ts2_1->add( BOOST_TEST_CASE( good_foo ) );
test_suite* ts_main_no_error = BOOST_TEST_SUITE( "Fake Test Suite Hierarchy no errors" );
ts_main_no_error->add( ts2_0 );
ts_main_no_error->add( BOOST_TEST_CASE( almost_good_foo ) );
ts_main_no_error->add( ts2_1 );
#define PATTERN_FILE_NAME "global-fixtures-test.pattern"
std::string pattern_file_name(
framework::master_test_suite().argc == 1
? (runtime_config::save_pattern() ? PATTERN_FILE_NAME : "./baseline-outputs/" PATTERN_FILE_NAME )
: framework::master_test_suite().argv[1] );
output_test_stream_for_loggers test_output( pattern_file_name, !runtime_config::save_pattern() );
// legacy API, we test that we catch exceptions in the ctor, and tests
// in the suite are running or not depending on that
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithCtor<&good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithCtor<&good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithCtor<&good_foo> >( test_output, ts_main_no_error, false);
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithCtor<&almost_good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithCtor<&almost_good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithCtor<&almost_good_foo> >( test_output, ts_main_no_error, false );
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithCtor<&bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithCtor<&bad_foo> >( test_output, ts_main, true ); // should fail the module
check_global_fixture< GlobalFixtureWithCtor<&bad_foo> >( test_output, ts_main_no_error, true );
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithCtor<&very_bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithCtor<&very_bad_foo> >( test_output, ts_main, true ); // should fail the module
check_global_fixture< GlobalFixtureWithCtor<&very_bad_foo> >( test_output, ts_main_no_error, true );
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithCtor<&very_bad_exception>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithCtor<&very_bad_exception> >( test_output, ts_main, true ); // should fail the module
check_global_fixture< GlobalFixtureWithCtor<&very_bad_exception> >( test_output, ts_main_no_error, true );
// here we test only for the setup function, tests should not be
// executed when setup fails, setup should be allowed to fail
// setup does not fail
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithSetup<&good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithSetup<&good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithSetup<&good_foo> >( test_output, ts_main_no_error, false );
// setup does not fail, with messages
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithSetup<&almost_good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithSetup<&almost_good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithSetup<&almost_good_foo> >( test_output, ts_main_no_error, false );
// setup fails
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithSetup<&bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithSetup<&bad_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithSetup<&bad_foo> >( test_output, ts_main_no_error, true );
// setup fails badly
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithSetup<&very_bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithSetup<&very_bad_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithSetup<&very_bad_foo> >( test_output, ts_main_no_error, true );
// setup fails with exception
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithSetup<&very_bad_exception>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithSetup<&very_bad_exception> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithSetup<&very_bad_exception> >( test_output, ts_main_no_error, true );
// here we test only for the teardown function, tests should not be
// executed when setup fails, setup should be allowed to fail
// teardown does not fail
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithTeardown<&good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithTeardown<&good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithTeardown<&good_foo> >( test_output, ts_main_no_error, false );
// teardown does not fail, with messages
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithTeardown<&almost_good_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithTeardown<&almost_good_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithTeardown<&almost_good_foo> >( test_output, ts_main_no_error, false );
// teardown fails
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithTeardown<&bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithTeardown<&bad_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithTeardown<&bad_foo> >( test_output, ts_main_no_error, true );
// teardown fails badly
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithTeardown<&very_bad_foo>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithTeardown<&very_bad_foo> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithTeardown<&very_bad_foo> >( test_output, ts_main_no_error, true );
// teardown fails with exception
test_output << "***********************" << std::endl;
test_output << "*********************** GlobalFixtureWithTeardown<&very_bad_exception>" << std::endl;
test_output << "***********************" << std::endl;
check_global_fixture< GlobalFixtureWithTeardown<&very_bad_exception> >( test_output, ts_main, true );
check_global_fixture< GlobalFixtureWithTeardown<&very_bad_exception> >( test_output, ts_main_no_error, true );
}

68
test/writing-test-ts/test-fixture-detect-setup-teardown-cpp11.cpp Normal file
View File

@@ -0,0 +1,68 @@
// (C) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
// checks issue https://svn.boost.org/trac/boost/ticket/5563 in particular
// the ability of the framework to detect new fixture signatures.
#define BOOST_TEST_MODULE test_fixture_detect_setup_teardown_cpp11
#include <boost/test/unit_test.hpp>
#include <iostream>
#include <boost/test/unit_test_suite.hpp>
#include <boost/test/framework.hpp>
using namespace boost::unit_test;
class fixture_without {
public:
fixture_without() {}
~fixture_without() {}
};
class fixture_with {
public:
fixture_with() {}
void setup() {}
void teardown() {}
~fixture_with() {}
};
class fixture_with_child : public fixture_with {
public:
fixture_with_child() {}
~fixture_with_child() {}
};
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect )
{
BOOST_CHECK(!impl_fixture::has_setup<fixture_without>::value);
BOOST_CHECK(!impl_fixture::has_setup<fixture_without>::value);
fixture_without obj;
setup_conditional(obj);
teardown_conditional(obj);
}
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect_both )
{
BOOST_CHECK(impl_fixture::has_setup<fixture_with>::value);
BOOST_CHECK(impl_fixture::has_setup<fixture_with>::value);
fixture_with obj;
setup_conditional(obj);
teardown_conditional(obj);
}
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect_both_from_child )
{
// should detect this with the C++11/declspec approach
BOOST_CHECK(impl_fixture::has_setup<fixture_with_child>::value);
BOOST_CHECK(impl_fixture::has_setup<fixture_with_child>::value);
fixture_with_child obj;
setup_conditional(obj);
teardown_conditional(obj);
}

72
test/writing-test-ts/test-fixture-detect-setup-teardown.cpp Normal file
View File

@@ -0,0 +1,72 @@
// (C) Copyright Raffi Enficiaud 2017.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
// See http://www.boost.org/libs/test for the library home page.
// checks issue https://svn.boost.org/trac/boost/ticket/5563 in particular
// the ability of the framework to detect new fixture signatures.
#define BOOST_TEST_MODULE test_fixture_detect_setup_teardown
#include <boost/test/unit_test.hpp>
#include <iostream>
#include <boost/test/unit_test_suite.hpp>
#include <boost/test/framework.hpp>
using namespace boost::unit_test;
class fixture_without {
public:
fixture_without() {}
~fixture_without() {}
};
class fixture_with {
public:
fixture_with() {}
void setup() {}
void teardown() {}
~fixture_with() {}
};
class fixture_with_child : public fixture_with {
public:
fixture_with_child() {}
~fixture_with_child() {}
};
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect )
{
BOOST_CHECK(!impl_fixture::has_setup<fixture_without>::value);
BOOST_CHECK(!impl_fixture::has_setup<fixture_without>::value);
fixture_without obj;
setup_conditional(obj);
teardown_conditional(obj);
}
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect_both )
{
BOOST_CHECK(impl_fixture::has_setup<fixture_with>::value);
BOOST_CHECK(impl_fixture::has_setup<fixture_with>::value);
fixture_with obj;
setup_conditional(obj);
teardown_conditional(obj);
}
#if defined(BOOST_NO_CXX11_DECLTYPE) || defined(BOOST_NO_CXX11_TRAILING_RESULT_TYPES)
BOOST_AUTO_TEST_CASE( fixture_setup_teardown_detect_both_from_child )
{
// cannot detect this with the C++03 approach
BOOST_CHECK(!impl_fixture::has_setup<fixture_with_child>::value);
BOOST_CHECK(!impl_fixture::has_setup<fixture_with_child>::value);
fixture_with_child obj;
setup_conditional(obj);
teardown_conditional(obj);
}
#endif