////
Copyright (c) 2024 The C++ Alliance, Inc. (https://cppalliance.org)

Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

Official repository: https://github.com/boostorg/website-v2-docs
////
= Introduction to Testing
:navtitle: Introduction

The Boost libraries are intended to be both reliable and portable. Every experienced programmer knows that means each library must be tested against a suitable number of test cases, on a range of platforms and compilers, and then tested again (_regression tested_) every time a change is made. The regression tests are run again before every public release.

There are three broad dimensions to Boost library testing:

. The actual test codes - unit tests, regression tests, new feature and bug tests. Refer to xref:testing/writing-tests.adoc[].
. Tests that are integrated with the primary build systems, xref:build-systems/b2.adoc[] and xref:build-systems/cmake.adoc[].
. xref:testing/continuous-integration.adoc[]

The new library developer needs to consider all three, however the initial focus on a new library will be the first in this list. Or, to put it another way, "Quality assurance based on a wide range of targeted tests" as one of the key answers to Professor Hoare's question:

"_How did software get so reliable without proof?_"

* <<Define a Test Matrix>>
* <<Use Predefined Macros from Boost.Config>>
* <<Regression Testing>>
* <<Next Steps>>
* <<See Also>>

== Define a Test Matrix

When you're looking to submit a library to the Boost collection, it's essential to ensure broad compatibility with various compilers, platforms, and configurations. Create a test matrix of what you intend to support, and document what you do not intend to support, and consider:

. Testing with multiple versions of each compiler. Popular compilers to consider include:
** GCC (GNU Compiler Collection)
** Clang
** MSVC (Microsoft Visual pass:[C++])
+
The Boost user base can be using older versions of these compilers, so strive for compatibility with a reasonable range of versions if possible.
+
If your library depends on other Boost libraries or external libraries, ensure they are compatible with the compilers you are targeting. Be clear about any dependencies or prerequisites in your documentation.
+
Be wary of non-standard compiler features or extensions. If you must use them, guard them with appropriate preprocessor checks.
Boost provides its own set of configuration macros to help with this.

. Compatibility with various versions of the pass:[C++] Standard: pass:[C++]11, pass:[C++]14, pass:[C++]17, pass:[C++]20, and pass:[C++]23. Some Boost libraries support many of the standards, while others target only more recent ones. 
+
Note:: Supporting pass:[C++]03 is no longer considered good practice.

. Different operating systems, including Linux, Windows, macOS, and others like various BSDs.

. Different architectures: x86, x64, ARM, MIPS. Architecture can affect word size (usually 32 or 64 bit), endianness, memory alignment, inline assembly, cache sizes, latency, and other memory and performance issues.

When you have outlined your test matrix, study the predefined macros available to assist you, and make adjustments to your matrix appropriately.

== Use Predefined Macros from Boost.Config

There are a set of macros in the boost:config[] library that can be used to identify compiler capabilities, platform specifics, and other configuration details. These macros are designed to smooth out differences between compilers and platforms, allowing for more portable code, for example:

=== Compiler Identification

* `BOOST_COMP_GNUC`: Defined if the compiler is GCC.
* `BOOST_COMP_MSVC`: Defined if the compiler is Microsoft Visual C++.

=== Compiler Version Checks

* `BOOST_COMP_MSVC >= BOOST_VERSION_NUMBER(19, 0, 0)`: Checks if the MSVC version is 19 or greater.

=== Platform Checks

* `BOOST_OS_LINUX`: Defined if the operating system is Linux.
* `BOOST_ARCH_X86_64`: Defined if the architecture is x86_64.

=== Standard Library Checks

* `BOOST_LIB_C_GNU`: Defined if the C standard library is from https://gcc.gnu.org/onlinedocs/libstdc++/[GNU].
* `BOOST_LIB_STD_DINKUMWARE`: Defined if the standard library is from https://docwiki.embarcadero.com/RADStudio/Sydney/en/Dinkumware_Standard_C%2B%2B_Library[Dinkumware] (often associated with MSVC).

=== Feature Checks

* `BOOST_NO_CXX14_RETURN_TYPE_DEDUCTION`: Defined if the compiler does not support return type deduction introduced in C++14.
* `BOOST_NO_CXX11_AUTO_DECLARATIONS`: Defined if the compiler does not support auto declarations from C++11. For example:
+
[source,cpp]
----
#include <boost/config.hpp>

// ...
#ifdef BOOST_NO_CXX11_AUTO_DECLARATIONS
// Use traditional type declaration
int x = 42;
#else
// Use C++11 auto
auto x = 42;
#endif
// ...
----
+
Use this same coding structure for any of the other macros.

== Regression Testing

Boost releases are run through regression tests which automatically generates compiler status HTML tables for various platforms. Unless otherwise indicated, the pass:[C++] Standard Library implementation is the one shipped with the compiler. Refer to https://www.boost.org/doc/regression/library_status/doc/library_status.html[Generating Library Status Tables].

Note:: The HTML tables are not a good indication of a particular compiler's compliance with the pass:[C++] Standard. The Boost libraries often contain workarounds which mask compiler deficiencies.
+
Some regression tests are run only occasionally, and might be relatively out-of-date. Check the date and revision in the column headings.

== Next Steps

First, familiarize yourself with the Boost xref:testing/test-policy.adoc[]. 

Then, read the documentation for the following libraries, which support the writing of unit, feature and regression tests:

* boost:test[]
* boost:core[]

Start small and develop a good understanding of how these testing libraries work, before writing more than a few tests.

=== Advanced Testing

When you have a good understanding of the basic testing procedures, look into more advanced techniques, such as xref:testing/fuzzing.adoc[].

== See Also

* xref:release-process.adoc[]
* xref:user-guide:ROOT:testing-debugging.adoc[]
* xref:testing/writing-tests.adoc[]
















