boost/test
2
0
Fork 0
mirror of https://github.com/boostorg/test.git synced 2026-01-28 07:42:12 +00:00
Code Issues Projects Releases Wiki Activity

Merge branch 'topic/BOOST_DATA_TEST_CASE_F' into develop

Browse Source 
Conflicts:
	doc/closing_chapters/change_log.qbk
This commit is contained in:
Raffi Enficiaud
2016-02-20 11:00:36 +01:00
parent 8e1d24188d 585acec903
commit bfaf02fa28
7 changed files with 110 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
doc/closing_chapters/change_log.qbk
View File

@@ -13,11 +13,14 @@
[h5 New features]
* now datasets support any [link boost_test.tests_organization.test_cases.test_case_generation.datasets arity], using the
variadic template support of the compiler.
* now datasets support fixtures through `BOOST_DATA_TEST_CASE_F`, see
[link boost_test.tests_organization.test_cases.test_case_generation.datasets here] for more details
[h5 Bugfixes and feature requests]
# [ticket 6767] Use of namespace qualifier with floating point exception functions breaks if they are macros
# [ticket 8905] boost/test/impl/debug.ipp: Ignores return value from WaitForSingleObject
# [ticket 9943] Runtime parameter Random seed for random order of test cases not respected correctly
# [ticket 11854] Add fixture support in `BOOST_DATA_TEST_CASE`
# [ticket 11887] `BOOST_TEST(3u == (std::max)(0u, 3u))` fails
# [ticket 11889] BOOST_DATA_TEST_CASE fails to compile for 4D and higher dimensional grids

1
doc/test.qbk
View File

@@ -103,6 +103,7 @@
[/ parametrized test cases]
[def __BOOST_PARAM_TEST_CASE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_case_parameter `BOOST_PARAM_TEST_CASE`]]
[def __BOOST_DATA_TEST_CASE__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_dataset `BOOST_DATA_TEST_CASE`]]
[def __BOOST_DATA_TEST_CASE_F__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_dataset_fixture `BOOST_DATA_TEST_CASE_F`]]
[def __BOOST_TEST_DATASET_MAX_ARITY__ [link boost_test.utf_reference.test_org_reference.test_org_boost_test_dataset `BOOST_TEST_DATASET_MAX_ARITY`]]
[/ test suites]

28
doc/test_organization/parametric_test_case_generation.qbk
View File

@@ -95,8 +95,8 @@ The facilities provided by the __UTF__ addressed the issues described above:
* the notion of *datasets* eases the description of the class of inputs for test cases. The datasets also implement several
operations that enable their combinations to create new, more complex datasets,
* a single macro, __BOOST_DATA_TEST_CASE__, is used for the declaration and registration of a test case over a collection
of values (samples),
* two macros, __BOOST_DATA_TEST_CASE__ and __BOOST_DATA_TEST_CASE_F__, respectively without and with fixture support,
are used for the declaration and registration of a test case over a collection of values (samples),
* each test case, associated to a unique value, is executed independently from others. These tests are guarded in the same
way regular test cases are, which makes the execution of the tests over each sample of a dataset isolated, robust,
repeatable and ease the debugging,
@@ -161,14 +161,19 @@ These operations result in new datasets, in which the samples are of an augmente
[/ ################################################################################################################################## ]
[section:datasets_auto_registration Declaring and registering test cases with datasets]
In order to declare and register a data-driven test-case, the macro __BOOST_DATA_TEST_CASE__ should be used. This macro is variadic and can be used
in the following forms:
In order to declare and register a data-driven test-case, the macros __BOOST_DATA_TEST_CASE__ or __BOOST_DATA_TEST_CASE_F__
should be used. Those two forms are equivalent, with the difference that `BOOST_DATA_TEST_CASE_F` supports fixtures.
Those macros are variadic and can be used in the following forms:
``
__BOOST_DATA_TEST_CASE__(test_case_name, dataset) { /* dataset1 of arity 1 */ }
BOOST_DATA_TEST_CASE(test_case_name, dataset, var1) { /* datasets of arity 1 */ }
BOOST_DATA_TEST_CASE(test_case_name, dataset, var1, ..., varN) { /* datasets of arity N */ }
__BOOST_DATA_TEST_CASE_F__(fixture, test_case_name, dataset) { /* dataset1 of arity 1 with fixture */ }
BOOST_DATA_TEST_CASE_F(fixture, test_case_name, dataset, var1) { /* dataset1 of arity 1 with fixture */ }
BOOST_DATA_TEST_CASE_F(fixture, test_case_name, dataset, var1, ..., varN) { /* dataset1 of arity N with fixture */ }
``
The first form of the macro is for datasets of arity 1. The value of the sample being executed by the test body is
@@ -184,25 +189,28 @@ of the variables `var1`, ..., `varN` corresponds to the index 1, ... `N` of the
[snippet_dataset1_3]
The next three forms of declaration, with `BOOST_DATA_TEST_CASE_F`, are equivalent to the previous ones, with the difference being in the support of
a fixture that is execute before the test body for each sample. The fixture should follow the expected interface as detailed
[link boost_test.tests_organization.fixtures.models here].
The arity of the dataset and the number of variables should be exactly the same, the first form being a short-cut for the
case of arity 1.
[tip A compilation-time check is performed on the coherence of the arity of the dataset and the number of variables `var1`... `varN`.
The maximal supported arity is controlled by the macro __BOOST_TEST_DATASET_MAX_ARITY__, that can be overridden prior to including the
__UTF__ headers.]
For compilers *without variadic template* support, the maximal supported arity is controlled by the macro
__BOOST_TEST_DATASET_MAX_ARITY__, that can be overridden /prior/ to including the __UTF__ headers.]
It should be emphasized that this macro does not declares a test case, such as __BOOST_AUTO_TEST_CASE__, but declares and
registers as many test cases as there are samples in the dataset given as argument. Each test case runs on exactly *one*
It should be emphasized that those macros do not declare a unique test case (as __BOOST_AUTO_TEST_CASE__ would do) but declare and
register as many test cases as there are samples in the dataset given in argument. Each test case runs on exactly *one*
sample of the dataset.
Exactly as regular test cases, each test - or each sample - is executed within the test body in a /guarded manner/:
* the test execution are independent: if an error occurs for one sample, the remaining samples execution is not affected
* in case of error, the [link boost_test.test_output.contexts context] within which the error occurred is reported in the [link boost_test.test_output log].
This context contains the sample for which the test failed, which would ease the debugging.
[caution The macro __BOOST_DATA_TEST_CASE__ is available only for compilers with support for variadic macros.]
[caution The macros __BOOST_DATA_TEST_CASE__ and __BOOST_DATA_TEST_CASE_F__ are available only for compilers with support for *variadic macros*.]
[endsect]

34
doc/test_organization/testorg_reference.qbk
View File

@@ -82,17 +82,45 @@ BOOST_DATA_TEST_CASE(test_case_name, dataset, var1, var2..., varN)
//...
BOOST_TEST(varN != 0);
}
``
same as the second form, but for dataset of arity `N`. The maximal arity (the maximal value of `N`) is controlled by the macro
same as the second form, but for dataset of arity `N`.
For compilers *lacking the variadic template* support, the maximal arity (the maximal value of `N`) is controlled by the macro
`BOOST_TEST_DATASET_MAX_ARITY` which is set to `10` by default. If you need a greater value, define `BOOST_TEST_DATASET_MAX_ARITY`
to the desired value before including the __UTF__ headers.
to the desired value *before* including the __UTF__ headers.
See [link boost_test.tests_organization.test_cases.test_case_generation.datasets_auto_registration here] for more details.
[endsect] [/section:test_org_boost_test_dataset]
[section:test_org_boost_test_dataset_fixture `BOOST_DATA_TEST_CASE_F`]
Declares and registers a data-driven test case, using a particular dataset and a fixture. This is basically the same as
__BOOST_DATA_TEST_CASE__ with fixture support added.
``
struct my_fixture {
my_fixture() : some_string("environment X") {
}
std::string some_string;
};
BOOST_DATA_TEST_CASE_F(my_fixture, test_case_name, dataset, var1, var2..., varN)
{
BOOST_TEST(var1 != 0);
//...
BOOST_TEST(varN != 0);
}
``
The fixture should implement the appropriate [link boost_test.tests_organization.fixtures.models interface].
As any fixture, it is possible to have test assertions in the fixture class.
See [link boost_test.tests_organization.fixtures.case here] for more details on fixtures and
[link boost_test.tests_organization.test_cases.test_case_generation.datasets_auto_registration here] for more details on datasets
declaration.
[endsect] [/section:test_org_boost_test_dataset_fixture]
[/ Test suites ###############################################################################################]
[section:test_org_boost_test_suite `BOOST_TEST_SUITE`]

5
doc/test_organization/testorg_summary.qbk
View File

@@ -45,6 +45,11 @@
[Declaration of the body of a test case on datasets]
]
[
[__BOOST_DATA_TEST_CASE_F__]
[Same as __BOOST_DATA_TEST_CASE__ with fixtures support]
]
[
[__BOOST_TEST_DATASET_MAX_ARITY__]
[Controlling the maximal arity of the data test case declared with __BOOST_DATA_TEST_CASE__]

42
include/boost/test/data/test_case.hpp
View File

@@ -165,18 +165,22 @@ make_test_case_gen( const_string tc_name, const_string tc_file, std::size_t tc_l
BOOST_PP_SEQ_FOR_EACH_I(BOOST_DATA_TEST_CASE_PARAM, _, params)) \
/**/
#define BOOST_DATA_TEST_CASE_IMPL( arity, test_name, dataset, params ) \
struct test_name { \
#define BOOST_DATA_TEST_CASE_IMPL(arity, F, test_name, dataset, params) \
struct test_name : public F { \
template<BOOST_PP_ENUM_PARAMS(arity, typename Arg)> \
static void test_method( BOOST_DATA_TEST_CASE_PARAMS( params ) ) \
{ \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" fixture entry.");\
test_name t; \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" entry."); \
BOOST_TEST_CONTEXT( "" \
BOOST_PP_SEQ_FOR_EACH(BOOST_DATA_TEST_CONTEXT, _, params)) \
_impl(BOOST_PP_SEQ_ENUM(params)); \
t._impl(BOOST_PP_SEQ_ENUM(params)); \
BOOST_TEST_CHECKPOINT('"' << #test_name << "\" exit."); \
} \
private: \
template<BOOST_PP_ENUM_PARAMS(arity, typename Arg)> \
static void _impl(BOOST_DATA_TEST_CASE_PARAMS( params )); \
void _impl(BOOST_DATA_TEST_CASE_PARAMS( params )); \
}; \
\
BOOST_AUTO_TU_REGISTRAR( test_name )( \
@@ -190,13 +194,13 @@ BOOST_AUTO_TU_REGISTRAR( test_name )( \
void test_name::_impl( BOOST_DATA_TEST_CASE_PARAMS( params ) ) \
/**/
#define BOOST_DATA_TEST_CASE_WITH_PARAMS( test_name, dataset, ... ) \
#define BOOST_DATA_TEST_CASE_WITH_PARAMS( F, test_name, dataset, ... ) \
BOOST_DATA_TEST_CASE_IMPL( BOOST_PP_VARIADIC_SIZE(__VA_ARGS__), \
test_name, dataset, \
F, test_name, dataset, \
BOOST_PP_VARIADIC_TO_SEQ(__VA_ARGS__) ) \
/**/
#define BOOST_DATA_TEST_CASE_NO_PARAMS( test_name, dataset ) \
BOOST_DATA_TEST_CASE_WITH_PARAMS( test_name, dataset, sample ) \
#define BOOST_DATA_TEST_CASE_NO_PARAMS( F, test_name, dataset ) \
BOOST_DATA_TEST_CASE_WITH_PARAMS( F, test_name, dataset, sample ) \
/**/
#if BOOST_PP_VARIADICS_MSVC
@@ -205,14 +209,32 @@ BOOST_AUTO_TU_REGISTRAR( test_name )( \
BOOST_PP_CAT( \
BOOST_PP_IIF(BOOST_PP_EQUAL(BOOST_PP_VARIADIC_SIZE(__VA_ARGS__),2), \
BOOST_DATA_TEST_CASE_NO_PARAMS, \
BOOST_DATA_TEST_CASE_WITH_PARAMS) (__VA_ARGS__), ) \
BOOST_DATA_TEST_CASE_WITH_PARAMS) ( \
BOOST_AUTO_TEST_CASE_FIXTURE, __VA_ARGS__), ) \
/**/
#define BOOST_DATA_TEST_CASE_F( F, ... ) \
BOOST_PP_CAT( \
BOOST_PP_IIF(BOOST_PP_EQUAL(BOOST_PP_VARIADIC_SIZE(__VA_ARGS__),2), \
BOOST_DATA_TEST_CASE_NO_PARAMS, \
BOOST_DATA_TEST_CASE_WITH_PARAMS) ( \
F, __VA_ARGS__), ) \
/**/
#else
#define BOOST_DATA_TEST_CASE( ... ) \
BOOST_PP_IIF(BOOST_PP_EQUAL(BOOST_PP_VARIADIC_SIZE(__VA_ARGS__),2), \
BOOST_DATA_TEST_CASE_NO_PARAMS, \
BOOST_DATA_TEST_CASE_WITH_PARAMS) (__VA_ARGS__) \
BOOST_DATA_TEST_CASE_WITH_PARAMS) ( \
BOOST_AUTO_TEST_CASE_FIXTURE, __VA_ARGS__) \
/**/
#define BOOST_DATA_TEST_CASE_F( F, ... ) \
BOOST_PP_IIF(BOOST_PP_EQUAL(BOOST_PP_VARIADIC_SIZE(__VA_ARGS__),2), \
BOOST_DATA_TEST_CASE_NO_PARAMS, \
BOOST_DATA_TEST_CASE_WITH_PARAMS) ( \
F, __VA_ARGS__) \
/**/
#endif

20
test/test-organization-ts/datasets-test/test_case_interface-test.cpp
View File

@@ -93,6 +93,8 @@ BOOST_DATA_TEST_CASE( test_case_interface_06, samples1 * samples2 * samples3, in
++index6;
}
//____________________________________________________________________________//
// test dataset dim > 3
int index7 = 0;
@@ -111,4 +113,22 @@ BOOST_DATA_TEST_CASE( test_case_interface_07, samples1 * samples2 * samples3 * s
++index7;
}
//____________________________________________________________________________//
static int index8 = 1;
struct SharedFixture {
SharedFixture()
: m_expected(index8++)
{
}
int m_expected;
};
BOOST_DATA_TEST_CASE_F( SharedFixture, test_case_interface_08, data::make({1,2,3}) )
{
BOOST_TEST( sample == m_expected );
}
// EOF