boost/boostlook
2
0
Fork 0
mirror of https://github.com/boostorg/boostlook.git synced 2026-02-25 16:22:12 +00:00
Code Issues Projects Releases Wiki Activity
Files
33ce117d70197f543c1957616a233cc343d2c168
boostlook/preview/contributor-guide/testing/continuous-integration.html

1040 lines
49 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1">
<style>html.fonts-loading{visibility:hidden;opacity:0}</style>
<script>document.documentElement.classList.add('fonts-loading');</script>
<link rel="preload" href="../../_/font/NotoSansDisplay.woff2" as="font" type="font/woff2" crossorigin="anonymous" />
<link rel="preload" href="../../_/font/NotoSansDisplay-Italic.woff2" as="font" type="font/woff2" crossorigin="anonymous" />
<link rel="preload" href="../../_/font/MonaspaceNeon-Var.woff2" as="font" type="font/woff2" crossorigin="anonymous" />
<link rel="preload" href="../../_/font/MonaspaceXenon-Var.woff2" as="font" type="font/woff2" crossorigin="anonymous" />
<script>
(function() {
'use strict';
var revealed = false;
var reveal = function() {
if (revealed) return;
revealed = true;
document.documentElement.classList.remove('fonts-loading');
};
setTimeout(reveal, 3000);
if (!('FontFace' in window) || !('fonts' in document)) {
setTimeout(reveal, 100);
return;
}
var uiRoot = '../../_';
var fonts = [
{
family: 'Noto Sans',
url: uiRoot + '/font/NotoSansDisplay.woff2',
descriptors: { style: 'normal', weight: '100 900', stretch: '62.5% 100%' }
},
{
family: 'Noto Sans',
url: uiRoot + '/font/NotoSansDisplay-Italic.woff2',
descriptors: { style: 'italic', weight: '100 900', stretch: '62.5% 100%' }
},
{
family: 'Monaspace Neon',
url: uiRoot + '/font/MonaspaceNeon-Var.woff2',
descriptors: { style: 'normal', weight: '400' }
},
{
family: 'Monaspace Xenon',
url: uiRoot + '/font/MonaspaceXenon-Var.woff2',
descriptors: { style: 'italic', weight: '400' }
}
];
var loadPromises = fonts.map(function(f) {
try {
var face = new FontFace(f.family, 'url("' + f.url + '")', f.descriptors);
return face.load().then(function(loaded) {
document.fonts.add(loaded);
return loaded;
}).catch(function() {
return null;
});
} catch (e) {
return Promise.resolve(null);
}
});
Promise.all(loadPromises)
.then(function() {
return document.fonts.ready;
})
.then(reveal)
.catch(reveal);
})();
</script> <title>Continuous Integration :: Boost Site Docs</title>
<link rel="canonical" href="https://boost.revsys.dev/contributor-guide/testing/continuous-integration.html">
<link rel="prev" href="sanitizers.html">
<link rel="next" href="fuzzing.html">
<meta name="generator" content="Antora 3.1.14">
<link rel="stylesheet" href="../../_/css/boostlook.css">
<link rel="stylesheet" href="../../_/css/site.css">
<link rel="stylesheet" href="../../_/css/vendor/tabs.css">
<script>
(function() {
if (window.self !== window.top) return;
var theme = localStorage.getItem('antora-theme');
if (!theme && window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
theme = 'dark';
}
if (theme === 'dark') document.documentElement.classList.add('dark');
})();
</script>
<script>var uiRootPath = '../../_'</script>
<link rel="icon" href="../../_/img/favicons/favicon.ico" type="image/x-icon">
<!-- Favicon configuration -->
<link rel="apple-touch-icon" sizes="180x180" href="../../_/img/favicons/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="../../_/img/favicons/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="../../_/img/favicons/favicon-16x16.png">
<link rel="manifest" href="../../_/img/favicons/site.webmanifest">
<link rel="shortcut icon" href="../../_/img/favicons/favicon.ico">
</head>
<body class="article toc2 toc-left">
<div class="boostlook">
<script type="module">import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs'; mermaid.initialize({"startOnLoad":true});</script> <div id="header">
<div id="toc" class="nav-container toc2" data-component="contributor-guide" data-version="">
<aside class="nav">
<button class="nav-close"></button>
<div class="panels">
<div class="nav-panel-menu is-active" data-panel="menu">
<nav class="nav-menu">
<div class="title-row">
<h3 class="title"><a href="../index.html">Contributor Guide</a></h3>
<button class="theme-toggle" aria-label="Toggle dark mode" title="Toggle theme" style="display:none">
<i class="fas fa-sun theme-icon-light"></i>
<i class="fas fa-moon theme-icon-dark"></i>
</button> </div>
<ul class="nav-list">
<ul class="nav-list">
<li class="" data-depth="1">
<a class="nav-link" href="../getting-involved.html">Getting Involved</a>
</li>
<li class="" data-depth="1">
<a class="nav-link" href="../contributors-faq.html">Contributors FAQ</a>
</li>
<li class="" data-depth="1">
<span class="nav-text">Requirements</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../requirements/library-requirements.html">Library</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../requirements/license-requirements.html">License</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../requirements/portability-requirements.html">Portability</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../requirements/organization-requirements.html">Organization</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../requirements/library-metadata.html">Metadata</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Design</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/design-best-practices.html">Best Practices</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/headers.html">Headers</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/backwards-compatibility.html">Backwards Compatibility</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/separate-compilation.html">Separate Compilation</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/dependencies.html">Dependencies</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../design-guide/borland.html">Borland Portability</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Development</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../version-control.html">Version Control</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../best-practices.html">Best Practices</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../debug-visualisers.html">Debug Visualizers</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Testing</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="intro.html">Introduction</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="test-policy.html">Test Policy</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="boost-test-matrix.html">Test Matrix</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="regression-tests.html">Local Regression Tests</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="writing-tests.html">Writing Tests</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="sanitizers.html">Sanitizers</a>
</li>
<li class=" is-current-page" data-depth="2">
<a class="nav-link" href="continuous-integration.html">Continuous Integration</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="fuzzing.html">Fuzzing</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">The Super-Project</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../superproject/overview.html">Layout</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../superproject/getting-started.html">Getting Started</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../superproject/library-maintenance.html">Library Maintenance</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../superproject/library-workflow.html">Library Workflow</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Writing Documentation</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../docs/layout.html">Guidelines</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../docs/content.html">Content</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../docs/components.html">Components</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../docs/antora.html">Antora Guide</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../docs/asciidoc.html">AsciiDoc Style Guide</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../docs/logo-policy-media-guide.html">Logo Policy and Media Guide</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Releases</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../release-process.html">Release Process</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../release-notes.html">Release Notes</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Contributor Community</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../contributor-community-introduction.html">Introduction</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../oversight-committee.html">Fiscal Sponsorship Committee</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../tweeting.html">Tweeting</a>
</li>
<li class="" data-depth="2">
<a class="nav-link" href="../site-docs-style-guide.html">Site-docs Style Guide</a>
</li>
</ul>
<li class="" data-depth="1">
<span class="nav-text">Appendices</span>
</li>
<ul class="nav-list">
<li class="" data-depth="2">
<a class="nav-link" href="../organization-guide.html">Organization Guide</a>
</li>
</ul>
</ul>
</ul>
</nav>
</div>
</div>
</aside>
</div>
</div> <div id="content">
<article class="doc max-width-reset">
<div class="toolbar" role="navigation">
<button class="nav-toggle"></button>
<nav class="breadcrumbs" aria-label="breadcrumbs">
<ul>
<li>
<a href="../index.html" aria-label="Home: Contributor Guide">
<svg xmlns="http://www.w3.org/2000/svg" width="1rem" height="1rem" viewBox="0 -960 960 960" fill="#000000" aria-hidden="true"><path d="M160-120v-480l320-240 320 240v480H560v-280H400v280H160Z"/></svg>
</a>
</li>
<li>Testing</li>
<li><a href="continuous-integration.html">Continuous Integration</a></li>
</ul>
</nav>
<div class="spirit-nav">
<a accesskey="p" href="sanitizers.html">
<span class="material-symbols-outlined" title="Previous: Sanitizers">arrow_back</span>
</a>
<a class="disabled" accesskey="u" aria-disabled="true" tabindex="-1">
<span class="material-symbols-outlined" title="Up:">arrow_upward</span>
</a>
<a accesskey="n" href="fuzzing.html">
<span class="material-symbols-outlined" title="Next: Fuzzing">arrow_forward</span>
</a>
</div></div>
<h1 class="page">Continuous Integration</h1>
<div id="preamble">
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p><a href="#_introduction">Introduction</a></p>
</li>
<li>
<p><a href="#_new_library_ci_framework">New Library CI Framework</a></p>
</li>
<li>
<p><a href="#_github_actions">GitHub Actions</a></p>
</li>
<li>
<p><a href="#_drone">Drone</a></p>
</li>
<li>
<p><a href="#_travis_ci">Travis CI</a></p>
</li>
<li>
<p><a href="#_appveyor">AppVeyor</a></p>
</li>
<li>
<p><a href="#_circleci">CircleCI</a></p>
</li>
<li>
<p><a href="#_azure_pipelines">Azure Pipelines</a></p>
</li>
<li>
<p><a href="#_coverage">Coverage</a></p>
</li>
<li>
<p><a href="#_test_with_popular_compilers">Test with Popular Compilers</a></p>
</li>
<li>
<p><a href="#_docker_containers">Docker Containers</a></p>
</li>
<li>
<p><a href="#_see_also">See Also</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_introduction"><a class="anchor" href="#_introduction"></a>Introduction</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The Boost project uses Continuous Integration(CI) to ensure the quality and integrity of its code. CI is the practice of merging all developers' working copies into a shared mainline several times a day. The main aim is to prevent integration issues, which can be identified and addressed as early as possible.</p>
</div>
<div class="paragraph">
<p>Boost uses several CI services for testing on different platforms and compilers. Many libraries use two or three of the systems described here, as does the <a href="https://github.com/boostorg/boost/tree/master">Super-project</a> itself.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Notes</dt>
<dd>
<p>It is a requirement for a new library submission to Boost to include an appropriate CI system. Refer to the examples for each CI system to better understand what is involved.</p>
<div class="paragraph">
<p>Currently, Boost <em>does not</em> use Continuous Deployment(CD) - the release schedule is fixed and libraries are not continuously updated. This is to ensure complete testing and code reliability before any release for public use.</p>
</div>
</dd>
</dl>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_new_library_ci_framework"><a class="anchor" href="#_new_library_ci_framework"></a>New Library CI Framework</h2>
<div class="sectionbody">
<div class="paragraph">
<p>A recommended process to start a new Boost library is to clone the contents of the <a href="https://github.com/boostorg/boost-ci/tree/master">boost-ci</a> repo. This repo contains the basic CI framework for a new library. Clone it, then adjust and edit it appropriately.</p>
</div>
<div class="paragraph">
<p>This repository contains scripts that enable CI with:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="#_github_actions">GitHub Actions</a></p>
</li>
<li>
<p><a href="#_drone">Drone</a></p>
</li>
<li>
<p><a href="#_travis_ci">Travis CI</a></p>
</li>
<li>
<p><a href="#_appveyor">AppVeyor</a></p>
</li>
<li>
<p><a href="#_circleci">CircleCI</a></p>
</li>
<li>
<p><a href="#_azure_pipelines">Azure Pipelines</a></p>
</li>
<li>
<p><a href="#_codecov_io">Codecov.io</a></p>
</li>
<li>
<p><a href="#_coverity_scan">Coverity Scan</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For step-by-step processes, and tables of supported compilers, refer to the <a href="https://github.com/boostorg/boost-ci/blob/master/README.md">boost-ci README.md</a>.</p>
</div>
<div class="sect2">
<h3 id="_testing_trouble_spots"><a class="anchor" href="#_testing_trouble_spots"></a>Testing Trouble Spots</h3>
<div class="paragraph">
<p>There are a few areas of CI testing that are fairly unique to Boost, and can present difficulties to newcomers:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>You test with the <a href="../version-control.html#_the_super-project_repository" class="xref page">Super-project</a>, cloning it and placing your library inside a submodule. This is unique to Boost, you might expect to consume Boost as a dependency, but this is not how it works.</p>
</li>
<li>
<p>You test with the <strong>develop</strong> or <strong>master</strong> branches of <a href="../version-control.html#_the_super_project_repository" class="xref page">the Super-project Repository</a>, not with the latest stable release.</p>
</li>
<li>
<p>The CMake workflows are not trivial, nor are they the usual CMake steps.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Refer to the sections:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="#_test_with_popular_compilers">Test with Popular Compilers</a></p>
<div class="ulist">
<ul>
<li>
<p><a href="#_windows">Windows</a></p>
</li>
<li>
<p><a href="#_linux">Linux</a></p>
</li>
<li>
<p><a href="#_macos">MacOS</a></p>
</li>
</ul>
</div>
</li>
<li>
<p><a href="#_docker_containers">Docker Containers</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>When tackling CI issues, it is expected that you will ask questions on the <a href="https://lists.boost.org/mailman/listinfo.cgi/boost">Boost developers mailing list</a>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_github_actions"><a class="anchor" href="#_github_actions"></a>GitHub Actions</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Boost has been incorporating <a href="https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions">GitHub Actions</a> into its testing workflows. This is a CI/CD system platform provided directly by GitHub. It can run tests on a variety of platforms and configurations. Here&#8217;s a basic outline of how GitHub Actions works for Boost:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>GitHub Actions uses YAML files stored in a directory called <code>.github/workflows/</code> at the root of the repository to define the build environment and steps. For instance, a workflow file might specify which operating systems and compilers to use, any dependencies to install, and the commands to run for building and testing the code.</p>
</li>
<li>
<p>When changes are pushed to the repository, or at scheduled intervals, GitHub Actions automatically initiates the actions defined in the workflow file. This might include building the project and running the test suite.</p>
</li>
<li>
<p>After the workflow runs, GitHub Actions reports the result. If any step in the workflow fails, the failure is reported, which helps developers to quickly identify and address issues. The status of each workflow run is displayed on the GitHub interface, allowing anyone to quickly check the status of the project.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Boost also uses GitHub Actions support for <em>matrix builds</em> (allowing Boost to run the same build steps on multiple combinations of operating systems, compilers, etc.), caching of dependencies to speed up builds, and the ability to create custom actions.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Note</dt>
<dd>
<p>As there are a lot of libraries under <code>boostorg</code>, the quota of GitHub Actions can be reached, and they can take some time to complete.</p>
</dd>
</dl>
</div>
<div class="sect2">
<h3 id="_example_github_workflows"><a class="anchor" href="#_example_github_workflows"></a>Example GitHub Workflows</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/align/blob/5ad7df63cd792fbdb801d600b93cad1a432f0151/.github/workflows/ci.yml">Align library ci.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/coroutine2/blob/d7e1c1c4abcf8c1e90097279e485edea0b253a80/.github/workflows/ci.yml">Coroutine2 ci.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/mp11/blob/ef7608b463298b881bc82eae4f45a4385ed74fca/.github/workflows/ci.yml">Mp11 library ci.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/url/blob/f06f595ae760b6626764b8a01e3f8197b4016d6f/.github/workflows/ci.yml">URL library ci.yml</a> - contains an extensive test matrix</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_drone"><a class="anchor" href="#_drone"></a>Drone</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://docs.drone.io/">Drone</a> is an open-source CI system built on container technology. Each step in a Drone build usually runs in its own Docker container, allowing it to use any language, tool, or service that can run inside a Docker container. This offers excellent environment consistency and isolation between steps. You can run your Drone pipelines locally on your machine for testing purposes before pushing changes to your repository.</p>
</div>
<div class="paragraph">
<p>Drone can use a simple YAML configuration file, <code>.drone.yml</code>, placed at the root of your git repository. However, as pipelines grow in complexity, managing them with YAML can become challenging. This is where <code>.drone.jsonnet</code> and .<code>drone.star</code> files come in, which are associated with the <a href="https://jsonnet.org/">Jsonnet</a> and <a href="https://github.com/bazelbuild/starlark">Starlark</a> scripting languages respectively. They both serve the purpose of creating more dynamic, reusable, and maintainable pipeline configurations.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://jsonnet.org/">Jsonnet</a> is a data templating language for app configuration. It is a superset of JSON and allows for custom scripting to dynamically produce JSON (and by extension, YAML). With <code>.drone.jsonnet</code>, you can create advanced pipeline configurations that aren&#8217;t feasible with static YAML files.</p>
</li>
<li>
<p><a href="https://github.com/bazelbuild/starlark">Starlark</a> is a Python-inspired language that was created by Google and is used for configuring Bazel build systems. Similar to Jsonnet, it allows you to create more complex and maintainable pipeline configurations.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Both Jsonnet and Starlark allow you to use logic like conditionals and loops in your configuration files, making them a powerful tool for complex CI/CD pipelines. If you have solid Python experience, for example, Starlark&#8217;s Python-like syntax might be a better choice. If you&#8217;re working in a JSON-heavy environment, Jsonnet might be more suitable.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Drone uses a file named <code>.drone.yml</code>, <code>.drone.jsonnet</code> or <code>.drone.star</code>, and a <code>.drone</code> folder, at the root of the repository to define the build pipeline, including the environment, build steps, and notification settings. The environment is typically a Docker container, allowing for a high degree of flexibility and customization.</p>
</li>
<li>
<p>When changes are pushed to the repository, Drone automatically runs the build pipeline defined in the <code>.drone</code> file. This involves building the software and running a suite of tests.</p>
</li>
<li>
<p>After the pipeline finishes, Drone reports the results. If any step fails, developers can be notified immediately, helping to catch and fix issues early. The status of each pipeline run can also be seen on the Drone dashboard and optionally on the GitHub interface.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Drone also includes support for matrix builds, secrets management (for handling sensitive data), and plugins (for extending functionality).</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Note</dt>
<dd>
<p>Drone will not be available to a new library developer until the project is whitelisted. This whitelisting is not required for <a href="#_github_actions">GitHub Actions</a> and for the other CI systems listed in this section. Refer to <a href="https://github.com/CPPAlliance/drone-ci">Drone-ci</a> for specific details.</p>
</dd>
</dl>
</div>
<div class="sect2">
<h3 id="_example_drone_jsonnet_files"><a class="anchor" href="#_example_drone_jsonnet_files"></a>Example .drone.jsonnet Files</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/smart_ptr/blob/13be03abf880cdb616d0597c38880f53f1b415b8/.drone.jsonnet">Smart Pointer library .drone.jsonnet</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/variant2/blob/e2546b70ca04d4263f7a5917815506e488b6920f/.drone.jsonnet">Variant2 library .drone.jsonnet</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/unordered/blob/9a7d1d336aaa73ad8e5f7c07bdb81b2e793f8d93/.drone.jsonnet">Unordered library .drone.jsonnet</a></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_example_drone_star_files"><a class="anchor" href="#_example_drone_star_files"></a>Example .drone.star Files</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/exception/blob/b039b4ea18ef752d0c1684b3f715ce493b778060/.drone.star">Exception library .drone.star</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/type_traits/blob/89f5011b4a79d91e42735670e39f72cb25c86c72/.drone.star">Type Traits library .drone.star</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/yap/blob/ae49bf2744586e6bd6c0cedff4500a58a4386860/.drone.star">Yap library .drone.star</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_travis_ci"><a class="anchor" href="#_travis_ci"></a>Travis CI</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://docs.travis-ci.com/user/for-beginners/">Travis CI</a> is used for testing on Linux and macOS environments. It is a hosted, distributed continuous integration service used to build and test software projects hosted at GitHub. Here&#8217;s the overall process:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Travis CI uses a file named <code>.travis.yml</code> in the root of the repository to define the build environment and the build steps. This file lists the operating systems and compilers to use, any dependencies to install, and the commands to run for building and testing the code.</p>
</li>
<li>
<p>Whenever changes are pushed to the repository on GitHub, Travis CI automatically initiates a build and runs the tests according to the instructions in <code>.travis.yml</code>. Boost libraries usually have extensive test suites, and Travis CI helps ensure that changes do not break existing functionality.</p>
</li>
<li>
<p>After each build, Travis CI reports the results. If the build or any tests fail, it can notify the developers so that they can fix the issue. On GitHub, the status of the latest build is shown next to each commit, so anyone can quickly see whether the current version of the code is passing all tests.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Boost also uses Travis CI&#8217;s features for more complex workflows, using the matrix feature to test code with multiple versions of compilers or dependencies, and uses stages to structure their build pipeline into phases like <strong>build</strong>, <strong>test</strong>, and <strong>deploy</strong>.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Note</dt>
<dd>
<p>Although a fair number of libraries use Travis CI, it is not recommended for new libraries, due to some instances of stalling test builds.</p>
</dd>
</dl>
</div>
<div class="sect2">
<h3 id="_example_travis_yml_files"><a class="anchor" href="#_example_travis_yml_files"></a>Example .travis.yml Files</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/coroutine2/blob/d7e1c1c4abcf8c1e90097279e485edea0b253a80/.travis.yml">Coroutine2 library .travis.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/fiber/blob/2cb72f5dcefdeffbb36636234e6ccb36282f8ae3/.travis.yml">Fiber library .travis.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/iostreams/blob/5fe4de84f863964f7573be1146f524886146a5d3/.travis.yml">IOStreams library .travis.yml</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_appveyor"><a class="anchor" href="#_appveyor"></a>AppVeyor</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://www.appveyor.com/docs/">Appveyor</a> is used for testing on Windows. It is a continuous integration service which can be configured to build projects for various systems, including MSVC, MinGW, and Cygwin. The overall process is:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>AppVeyor uses a file named <code>appveyor.yml</code> in the root of the repository to define the build environment and the steps for building and testing. This file describes which Windows images to use, any dependencies that need to be installed, and the commands to run for building and testing the code.</p>
</li>
<li>
<p>When changes are pushed to the GitHub repository, AppVeyor automatically initiates a build and runs the tests according to the instructions in <code>appveyor.yml</code>. The goal of this is to catch and fix any failures or issues that occur in the Windows environment.</p>
</li>
<li>
<p>After each build, AppVeyor reports the result. If the build or any tests fail, it notifies the developers, allowing them to address the issues. The status of the latest build can also be seen on GitHub, providing an at-a-glance view of the code&#8217;s health.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>AppVeyor also supports parallel testing, a build cache to speed up builds, and the ability to deploy built artifacts.</p>
</div>
<div class="sect2">
<h3 id="_example_appveyor_yml_files"><a class="anchor" href="#_example_appveyor_yml_files"></a>Example appveyor.yml Files</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/beast/blob/c316c6bd3571991aeac65f0fc35fca9067bc7906/appveyor.yml">Beast library appveyor.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/iostreams/blob/5fe4de84f863964f7573be1146f524886146a5d3/appveyor.yml">IOStreams library appveyor.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/mp11/blob/ef7608b463298b881bc82eae4f45a4385ed74fca/appveyor.yml">Mp11 library appveyor.yml</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_circleci"><a class="anchor" href="#_circleci"></a>CircleCI</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://circleci.com/developer">CircleCI</a> is a CI/CD platform that supports a wide range of languages, tools, and services, making it flexible for different testing requirements. It is less commonly used than <a href="#_github_actions">GitHub Actions</a> or <a href="#_appveyor">AppVeyor</a>, but is used by <a href="../version-control.html#_the_super_project_repository" class="xref page">the Super-project Repository</a> and a few libraries.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>CircleCI uses a file named <code>config.yml</code> stored in a directory called <code>.circleci</code> at the root of the repository. This file defines the build environment and steps, such as which Docker images to use, dependencies to install, and the commands for building and testing.</p>
</li>
<li>
<p>Upon changes being pushed to the repository or on a schedule, CircleCI automatically executes the instructions in the <code>config.yml</code> file. This usually includes building the project and running the test suite.</p>
</li>
<li>
<p>After the workflow completes, CircleCI reports the results. If any part of the workflow fails, developers are notified, which allows them to address the issues swiftly. The status of the workflow run is visible on the GitHub interface, providing at-a-glance insights into the project&#8217;s health.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>CircleCI also supports parallel testing, caching of dependencies, and matrix builds.</p>
</div>
<div class="sect2">
<h3 id="_example_config_yml_files"><a class="anchor" href="#_example_config_yml_files"></a>Example config.yml Files</h3>
<div class="paragraph">
<p>Refer to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/beast/blob/c316c6bd3571991aeac65f0fc35fca9067bc7906/.circleci/config.yml">Beast library config.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/geometry/blob/2ec9d65d1294edb97157b564726fdf56b6ac562f/.circleci/config.yml">Geometry library config.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/multiprecision/blob/380aae3c28c646ea2ca1b42156d83732295082d7/.circleci/config.yml">Multiprecision library config.yml</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_azure_pipelines"><a class="anchor" href="#_azure_pipelines"></a>Azure Pipelines</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops">Azure Pipelines</a> is a cloud service provided by Microsoft to automatically build, test, and deploy applications. Here&#8217;s how it generally works:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Azure Pipelines uses a file named <code>.azure-pipelines.yml</code> at the root of the repository to define the build environment and steps. This file specifies the operating systems and compilers to use, any dependencies to install, and the commands to run for building and testing the code.</p>
</li>
<li>
<p>When changes are pushed to the repository, Azure Pipelines automatically triggers a build and runs the tests according to the instructions in the <code>azure-pipelines.yml</code> file. This helps ensure that changes do not break existing functionality.</p>
</li>
<li>
<p>After each build, Azure Pipelines reports the results. If the build or any tests fail, it notifies the developers, allowing them to address the issues. The status of the latest build can also be seen on GitHub, providing an at-a-glance view of the code&#8217;s health.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Azure Pipelines provides several additional features, such as support for parallel testing, a build cache to speed up builds, and the ability to deploy built artifacts.</p>
</div>
<div class="sect2">
<h3 id="_example_azure_pipelines_yml_files"><a class="anchor" href="#_example_azure_pipelines_yml_files"></a>Example azure-pipelines.yml Files</h3>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/boost-ci/blob/master/.azure-pipelines.yml">boost-ci/.azure-pipelines.yml</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_coverage"><a class="anchor" href="#_coverage"></a>Coverage</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Code coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs. A program with high code coverage, measured as a percentage, has had more of its source code executed during testing, which generally means it has a lower chance of containing undetected bugs compared to a program with low code coverage.</p>
</div>
<div class="paragraph">
<p>Code coverage analysis can uncover areas of a library not covered by existing tests, so developers can write new tests to cover these blind spots. It can also highlight areas of over-testing, where the same code is tested redundantly, which could lead to slower test times without providing extra benefit.</p>
</div>
<div class="paragraph">
<p>There are several types of code coverage, including:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>Function Coverage</strong> measures if each function or method in the codebase has been called.</p>
</li>
<li>
<p><strong>Statement Coverage</strong> measures if each statement or instruction in the codebase has been executed.</p>
</li>
<li>
<p><strong>Branch Coverage</strong> measures if each possible branch from each decision point has been executed. For example, both true and false branches from an <code>if</code> statement.</p>
</li>
<li>
<p><strong>Condition Coverage</strong> measures if each boolean sub-expression has been evaluated to both true and false. For example, given <code>if (A==B || C==D)</code>, there are two boolean sub-expressions to evaluate.</p>
</li>
<li>
<p><strong>Path Coverage</strong> measures if all possible paths (sequence of statements, branches) have been covered. This is generally considered the most comprehensive, but also the most challenging to achieve, especially in complex programs.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In a CI pipeline, code coverage is typically measured (by <a href="#_codecov_io">Codecov.io</a> or <a href="#_coverity_scan">Coverity Scan</a>) after each change to the codebase. Coverage reports can be generated and reviewed to spot areas of the code that are not well-tested.</p>
</div>
<div class="sect2">
<h3 id="_codecov_io"><a class="anchor" href="#_codecov_io"></a>Codecov.io</h3>
<div class="paragraph">
<p><a href="https://about.codecov.io/">Codecov.io</a> is a tool that provides insights about code coverage in a software project. Code coverage is a measure of how much of your code is actually executed when your test suite runs. By highlighting parts of your code that aren&#8217;t tested, code coverage tools like Codecov help you write better tests and thus improve the quality of your software.</p>
</div>
<div class="paragraph">
<p>Here&#8217;s an overview of how Codecov works in the context of a CI pipeline:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Codecov integrates with GitHub. When you push code to your repository or create a pull request, it triggers your CI pipeline. Codecov uses a <code>.codecov.yml</code> (or <code>codecov.yml</code>) file to manage its settings. It&#8217;s placed at the root of your repository.</p>
<div class="ulist">
<ul>
<li>
<p>You can set minimum coverage thresholds that must be met, and configure how Codecov should behave if the thresholds aren&#8217;t met. For example, you might want Codecov to fail the status checks if the coverage drops by a certain percentage.</p>
</li>
<li>
<p>You can specify files or directories that should be ignored by Codecov. And you can customize the comments that Codecov makes on your pull requests. For example, you can change the layout of the comment, or disable comments entirely.</p>
</li>
<li>
<p>Codecov flags allow you to segregate coverage reports for different parts of your project or for different types of tests. Flags can be useful for projects that have multiple test suites or modules. <code>Carryforward</code> Flags help to handle reports for parts of the project that are not included in every CI run.</p>
</li>
</ul>
</div>
</li>
<li>
<p>In your CI pipeline, after your tests run, you&#8217;ll generate a coverage report. The report is in XML or JSON format.</p>
</li>
<li>
<p>The generated coverage report is then uploaded to Codecov. This is usually done by a command-line tool provided by Codecov, which you&#8217;ll add as a step in your CI pipeline. The tool takes care of finding the report, compressing it, and sending it to Codecov&#8217;s servers.</p>
</li>
<li>
<p>Codecov processes the uploaded report and provides detailed coverage information on its dashboard. It shows overall project coverage, coverage changes over time, coverage for individual files, and more. Codecov can also comment on pull requests, showing how the changes would affect overall coverage.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Codecov also provides a browser extension that overlays coverage data directly on GitHub, so you can see coverage information as you browse your code.</p>
</div>
<div class="sect3">
<h4 id="_example_codecov_yml_files"><a class="anchor" href="#_example_codecov_yml_files"></a>Example .codecov.yml Files</h4>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/beast/blob/c316c6bd3571991aeac65f0fc35fca9067bc7906/.codecov.yml">Beast library .codecov.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/date_time/blob/85e637cb325208c2af9af791c3a1948b4888c6cd/.codecov.yml">Date-time library .codecov.yml</a></p>
</li>
<li>
<p><a href="https://github.com/boostorg/json/blob/0a7860fcfce7d66c0abe3d96f666540c00c33f73/.codecov.yml">Json library .codecov.yml</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_coverity_scan"><a class="anchor" href="#_coverity_scan"></a>Coverity Scan</h3>
<div class="paragraph">
<p><a href="https://scan.coverity.com/">Coverity Scan</a> is a static analysis tool that detects defects and vulnerabilities in your source code. It is provided as a free service for open source projects, but there&#8217;s also a commercial version for private projects. Here&#8217;s a general workflow of how you can use Coverity Scan:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>First, you need to register your project with Coverity Scan. This involves providing some basic information about your project and agreeing to their terms of service.</p>
</li>
<li>
<p>The next step is to build your code and upload it to the Coverity Scan servers. This is typically done in your development environment, and there are a few steps involved:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Install the Coverity Scan Tool. This tool is used to "build" your code and analyze it for defects.</p>
</li>
<li>
<p>Instead of building with your usual build tool (like CMake or Maven), you build with the Coverity tool. This produces a file that contains all the information Coverity needs to analyze your code.</p>
</li>
<li>
<p>You then upload this file to the Coverity servers. You can automate this step as part of your CI pipeline.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Once your code is uploaded, Coverity analyzes it for defects and vulnerabilities. This process can take some time, depending on the size of your codebase.</p>
</li>
<li>
<p>Once the analysis is complete, you can review the results on the Coverity Scan website. Defects are categorized by type and severity, and you can drill down to see the exact lines of code that are affected.</p>
</li>
<li>
<p>Based on the results, you can then fix the defects in your code. After making changes, you&#8217;ll typically run the Coverity Scan process again to verify the fixes and find any new defects.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Coverity Scan is a powerful tool that can help improve the quality of your code. It&#8217;s particularly good at finding complex defects that are hard to catch with regular testing. However, it does require some setup and learning to use effectively, particularly when integrating it with a CI pipeline.</p>
</div>
<div class="sect3">
<h4 id="_example_coverity_calls"><a class="anchor" href="#_example_coverity_calls"></a>Example Coverity Calls</h4>
<div class="paragraph">
<p>Coverity Scan does not directly use a .yml or .yaml file for configuration like the other CI tools discussed here. Instead, Coverity Scan primarily relies on the build commands and Coverity Scan command-line tools to analyze the source code. You include the necessary Coverity Scan commands within the .yml files of your other CI tools, for example:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/beast/blob/f9433d22d0662a89a6cf1b84a214680cfd384e3f/.drone.star#L25">Beast library .drone.star</a> line 25, and <a href="https://github.com/boostorg/beast/blob/f9433d22d0662a89a6cf1b84a214680cfd384e3f/.drone/drone.sh#L134">Beast library drone.sh</a> line 134</p>
</li>
<li>
<p><a href="https://github.com/boostorg/json/blob/0a7860fcfce7d66c0abe3d96f666540c00c33f73/.drone.star#L58">Json library .drone.star</a> line 58, and <a href="https://github.com/boostorg/json/blob/0a7860fcfce7d66c0abe3d96f666540c00c33f73/.drone/drone.sh#L110">Json library drone.sh</a> line 110</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_test_with_popular_compilers"><a class="anchor" href="#_test_with_popular_compilers"></a>Test with Popular Compilers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Your CI test matrix should include one or more of the most popular compilers for each supported OS.</p>
</div>
<div class="sect2">
<h3 id="_windows"><a class="anchor" href="#_windows"></a>Windows</h3>
<div class="ulist">
<ul>
<li>
<p><a href="https://visualstudio.microsoft.com/downloads/">Microsoft Visual C++</a> (MSVC): This is Microsoft&#8217;s own compiler that comes with Visual Studio. It has excellent support for Windows-specific development and great debugging tools.</p>
</li>
<li>
<p><a href="https://sourceforge.net/projects/mingw/">MinGW - Minimalist GNU for Windows</a>: MinGW includes a port of the GCC (GNU Compiler Collection), which includes a C++ compiler. It&#8217;s useful for open-source projects and cross-platform development.</p>
</li>
<li>
<p><a href="https://clang.llvm.org/">Clang</a>: Clang is a compiler front end for the C, C++, and Objective-C programming languages. It uses LLVM as its back end and has been part of the LLVM release cycle since LLVM 2.6.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_linux"><a class="anchor" href="#_linux"></a>Linux</h3>
<div class="ulist">
<ul>
<li>
<p><a href="https://gcc.gnu.org/">GCC, the GNU Compiler Collection</a>: GCC is one of the most popular compilers for Linux. It supports multiple programming languages but is most often used as a C++ compiler. It&#8217;s open-source and is the default compiler on most Linux distributions.</p>
</li>
<li>
<p><a href="https://clang.llvm.org/">Clang</a>: Clang, part of the LLVM project, is a C++ compiler that provides a number of advantages over GCC, such as faster compile times and improved performance. It&#8217;s also known for providing more understandable compile errors.</p>
</li>
<li>
<p><a href="https://www.intel.com/content/www/us/en/developer/articles/news/intel-c-compiler-classic-2021-2-1-release.html">Intel Compiler</a>: While not as common for general use as GCC or Clang, the Intel C++ Compiler can produce highly optimized code, especially for parallel computation and vector operations. It&#8217;s often used in high-performance computing scenarios.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_macos"><a class="anchor" href="#_macos"></a>MacOS</h3>
<div class="ulist">
<ul>
<li>
<p>Clang is the default compiler for macOS and is provided with <a href="https://developer.apple.com/xcode/resources/">Xcode</a>, Apple&#8217;s integrated development environment. It&#8217;s known for providing more understandable compile errors and faster compile times compared to GCC.</p>
</li>
<li>
<p><a href="https://gcc.gnu.org/">GCC, the GNU Compiler Collection</a>: While not the default, GCC can also be used on macOS. It&#8217;s typically installed via a package manager like Homebrew. However, it&#8217;s worth noting that when you install GCC on a Mac, the default "gcc" command often still points to Clang for compatibility reasons, so you might need to use a version-specific command like "gcc-9" to use the real GCC.</p>
</li>
<li>
<p><a href="https://www.intel.com/content/www/us/en/developer/articles/news/intel-c-compiler-classic-2021-2-1-release.html">Intel Compiler</a>: The Intel C++ Compiler is also available on macOS and can produce highly optimized code, especially for parallel computation and vector operations. Like on Linux, it&#8217;s often used in high-performance computing scenarios.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_docker_containers"><a class="anchor" href="#_docker_containers"></a>Docker Containers</h2>
<div class="sectionbody">
<div class="paragraph">
<p><a href="https://docs.docker.com/get-docker/">Docker</a> can be used to provide isolation, which can be very useful with certain development environments. For example, when there is a need to replicate an environment which could not be replicated otherwise. For example, we use Ubuntu 16 frequently, but there&#8217;s no GitHub image for it. As a workaround, the Ubuntu 22 image is used, and a Ubuntu 16 Docker container is run on it.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_see_also"><a class="anchor" href="#_see_also"></a>See Also</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/boostorg/cmake">CMake Support</a></p>
</li>
<li>
<p><a href="sanitizers.html" class="xref page">Sanitize Your Code</a></p>
</li>
<li>
<p><a href="../version-control.html" class="xref page">Version Control</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="edit-this-page">
<a href="file:///Users/julio/dev/website-v2-docs/contributor-guide/modules/ROOT/pages/testing/continuous-integration.adoc">Edit this Page</a>
</div>
<nav class="pagination">
<span class="prev"><a href="sanitizers.html">Sanitizers</a></span>
<span class="next"><a href="fuzzing.html">Fuzzing</a></span>
</nav>
</article>
</div>
<div id="footer">
<script id="site-script" src="../../_/js/site.js" data-ui-root-path="../../_"></script>
<script async src="../../_/js/vendor/highlight.js"></script>
<script async src="../../_/js/vendor/tabs.js" data-sync-storage-key="preferred-tab"></script>
</div>
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink