test/doc/testing_tools/boost_test_collection_comparison.qbk

[/
 / Copyright (c) 2015 Boost.Test contributors
 /
 / Distributed under the Boost Software License, Version 1.0. (See accompanying
 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
 /]

[section:collections Collections comparison]

Instead of comparing a single value against another, there is often a need for comparing /collections/ of values.
A collection and indirectly the values it contains may be considered in several ways:

* collection as a /sequence of values/: this is the case for instance when `N` values are stored in a
  container. Containers in this case are used for storing several values, and iterating over the containers yields
  sequences that can be compared *element-wise*. The iteration should be in an order that is /a priori/ known
  [footnote this might not be the case
  for e.g. `std::unordered_map`, for which the buckets might be filled differently depending on the insertion order.],
  for being able to compare the sequences. The values in the collection are independent each other, and subsets can be compared as well.
* collection as an /ensemble/: this is the case where the elements of the collection define an /entity/,
  and no element can be dissociated from the others.
  An example would be a collection of letters for a specific word in the natural language; in this settings
  any of the character in the word/collection depends /semantically/ on the other and it is not possible to take
  a subset of it without breaking the meaning of the word. Another example would be a vector of size `N` representing a
  point in a `N` dimensional space, compared to another point with the relation "`<`": the comparison is application
  specific and a possible comparison would be the lexicographical ordering
  [footnote in this case `v_a < v_b` means that the point `v_a` is inside the rectangle (origin, `v_b`)].

The following observations can be done:

* the methods employed for comparing collections should be chosen adequately with the meaning of the collection,
* comparing sequences *element-wise* often involves writing loops in the test body, and if a dedicated tool is already in place
  the test body would gain in clarity and expressiveness (including the report in case of failure),
* some comparison methods such as the lexicographical one, have good general behavior (e.g. total ordering,
  defined for collections of different size), but are sometimes inappropriate.

__BOOST_TEST__ provides specific tools for comparing collections:

* using the /native/ [footnote either defined by the container or by the user] operator of the container of the collection,
  which is mentioned as the [link boost_test_coll_default_comp /default behavior/].
* using [link boost_test_coll_perelement element-wise] comparison for which extended failure diagnostic is provided,
* and using [link boost_test_coll_default_lex lexicographical] comparison for which extended failure diagnostic is provided,

More details about the concept of /collection/ in the __UTF__ is given [link what_is_a_collection /here/].

[#boost_test_coll_default_comp][h3 Default comparison]
The default comparison dispatches to the existing overloaded operator. Given two containers `c_a` and `c_b`,

``
BOOST_TEST(c_a op c_b)
``

is equivalent, in terms of test success, to

``
auto result = c_a op c_b;
BOOST_TEST(result);
``

In the example below, `operator==` is not defined for `std::vector` of different types, and the program would fail to
compile if the corresponding lines were uncommented (`std::vector` uses lexicographical comparison by default).

[bt_example boost_test_container_default..BOOST_TEST containers comparison default..run-fail]

[#boost_test_coll_perelement][h3 Element-wise comparison]
By specifying the manipulator [classref boost::test_tools::per_element], the comparison of the elements of the containers
are performed /element-wise/, in the order given by the forward iterators of the containers. This is a comparison on
the /sequences/ of elements generated by the containers, for which the __UTF__ provides advanced diagnostic.

In more details, let `c_a = (a_1,... a_n)` and `c_b = (b_1,... b_n)` be two sequences of same length, but not necessarily of same type.
Those sequences correspond to the content of the respective containers, in the order given by their iterator. Let
`op` be one of the [link boost_test_statement_overloads binary comparison operators].

``
BOOST_TEST(c_a op c_b, boost::test_tools::per_element() );
``

is equivalent to

``
if(c_a.size() == c_b.size())
{
  for(int i=0; i < c_a.size(); i++)
  {
    __BOOST_TEST_CONTEXT__("index" << i);
    BOOST_TEST(a_i op b_i);
  }
}
else
{
  BOOST_TEST(c_a.size() == c_b.size());
}
``

[warning this is fundamentally different from using the containers' default comparison operators (default behavior).]
[warning this is not an order relationship on containers. As a side effect, it is possible to have
 ``BOOST_TEST(c_a == c_b)`` and ``BOOST_TEST(c_a != c_b)`` failing at the same time]

Sequences are compared using the specified operator `op`, evaluated on the left and right elements of the respective sequences.
The order of the compared elements is given by the iterators of the respective containers [footnote the containers should yield the same
sequences for a fixed set of elements they contain].
In case of failure, the indices of the elements failing `op` are returned.

[bt_example boost_test_sequence_per_element..BOOST_TEST sequence comparison..run-fail]

[h4 Requirements]
For the sequences to be comparable element-wise, the following conditions should be met:

* the containers should yield the same number of elements,
* `op` should be one of the comparison operator `==`, `!=`, `<`, `<=`, `>`, `>=`
* the `a_i op b_i` should be defined, where the type of `a_i` and `b_i` are the type returned by the dereference operator
  of the respective collections.

[caution the resulting type of "`c_a == c_b`" is an [classref boost::test_tools::assertion_result assertion_result]: it is not
 possible to compose more that one comparison on the BOOST_TEST statement:

 ``
   BOOST_TEST(c_a == c_b == 42, boost::test_tools::per_element() ); // does not compile
 ``]

[#boost_test_coll_default_lex][h3 Lexicographic comparison]
By specifying the manipulator [classref boost::test_tools::lexicographic], the containers are compared using the /lexicographical/
order and for which the __UTF__ provides additional diagnostic in case of failure.

``
BOOST_TEST(c_a op c_b, boost::test_tools::lexicographic() );
``

The comparison is performed in the order given by forward iterators of the containers.

[tip lexicographic comparison yields a total order on the containers: the statements `c_a < c_b` and
 `c_b <= c_a` are mutually exclusive.]

[bt_example boost_test_container_lex..BOOST_TEST container comparison using lexicographical order..run-fail]

[/ TODO ]
[/ [h4 Making lexicographic comparison the default] ]
[/ TODO ]

[h4 Requirements]

* the containers should be of the exact same type
* `op` should be one of the ordered comparison operator `<`, `<=`, `>`, `>=`


[#what_is_a_collection][h3 What is a sequence?]
A sequence is given by the iteration over a /forward iterable/ container. A forward iterable container is a container (C++11):

* that implements the member functions `size` and `begin`, as well as the fields `const_iterator` and `value_type`
* and for which `value_type` is not of type `char` or `wchar_t`

To that respect, C-arrays are *not* forward iterable containers:
[bt_example boost_test_macro_container_c_array..BOOST_TEST C-arrays..run-fail]

The detection of the containers is delegated to the class [classref boost::unit_test::is_forward_iterable], which for C++11
detects the required member functions and fields. However for C++03, the types providing the sequences should be explicitly indicated
to the __UTF__ by a specialization of [classref boost::unit_test::is_forward_iterable]
[footnote Standard containers of the `STL` are recognized as collections.].

[endsect] [/ sequences]