mirror of
https://github.com/boostorg/build.git
synced 2026-01-26 06:22:26 +00:00
64d934d4a0
* test/TestCmd.py: Cause the names of preserved directories to be
printed.
* test/test_system.html: Document "PRESERVE*" environmental variables
and "pass_test"/"fail_test" methods.
[SVN r14389]
481 lines
17 KiB
HTML
481 lines
17 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
|
|
"http://www.w3.org/TR/html4/strict.dtd">
|
|
|
|
<html>
|
|
<head>
|
|
<meta name="generator" content=
|
|
"HTML Tidy for Linux/x86 (vers 1st March 2002), see www.w3.org">
|
|
<!--tidy options: -i -wrap 78 -->
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
|
|
<title>A testing system for Boost.Build</title>
|
|
<style type="text/css">
|
|
hr { color: black }
|
|
p.revision { text-align: right; font-style: italic }
|
|
pre.code { margin-left: 2em }
|
|
pre.output { margin-left: 2em }
|
|
img.banner { border: 0; float: left }
|
|
h1 { text-align: right }
|
|
br.clear { clear: left }
|
|
div.attention { color: red }
|
|
</style>
|
|
</head>
|
|
|
|
<body>
|
|
<p><a href="../../../index.htm"><img class="banner" height="86" width=
|
|
"277" alt="C++ Boost" src="../../../c++boost.gif"></a></p>
|
|
|
|
<h1>A testing system for Boost.Build<br class="clear">
|
|
</h1>
|
|
<hr>
|
|
|
|
<dl class="page-index">
|
|
<dt><a href="#sec-intro">Introduction and examples</a></dt>
|
|
|
|
<dd>
|
|
<dl class="page-index">
|
|
<dt><a href="#sec-intro-changing">Changing the working
|
|
directory</a></dt>
|
|
|
|
<dt><a href="#sec-intro-examining">Examining the working directory
|
|
and changing it</a></dt>
|
|
|
|
<dt><a href="#sec-intro-results">Test result</a></dt>
|
|
</dl>
|
|
</dd>
|
|
|
|
<dt><a href="#sec-reference">Reference documentation</a></dt>
|
|
|
|
<dd>
|
|
<dl class="page-index">
|
|
<dt><a href="#method-__init__">Method __init__</a></dt>
|
|
|
|
<dt><a href="#method-set_tree">Method <tt>set_tree</tt></a></dt>
|
|
|
|
<dt><a href="#method-write">Method <tt>write</tt></a></dt>
|
|
|
|
<dt><a href="#method-copy">Method <tt>copy</tt></a></dt>
|
|
|
|
<dt><a href="#method-touch">Method <tt>touch</tt></a></dt>
|
|
|
|
<dt><a href="#method-run_build_system">Method
|
|
<tt>run_build_system</tt></a></dt>
|
|
|
|
<dt><a href="#method-read">Method <tt>read</tt></a></dt>
|
|
|
|
<dt><a href="#method-read_and_strip">Method
|
|
<tt>read_and_strip</tt></a></dt>
|
|
|
|
<dt><a href="#methods-expectations">Methods for declaring
|
|
expectations</a></dt>
|
|
|
|
<dt><a href="#methods-ignoring">Methods for ignoring
|
|
changes</a></dt>
|
|
|
|
<dt><a href="#methods-result">Methods for explicitly specifying
|
|
results</a></dt>
|
|
|
|
<dt><a href="#class-list">Helper class <tt>List</tt></a></dt>
|
|
</dl>
|
|
</dd>
|
|
</dl>
|
|
<hr>
|
|
|
|
<h2><a name="sec-intro">Introduction and examples</a></h2>
|
|
|
|
<p>The testing system for Boost.Build is derived from the
|
|
<tt>TestCmd</tt> Python module used for testing the <a href=
|
|
"http://www.scons.org">Scons</a> build tools. The changes are very few,
|
|
and I hope that some of them may be adopted by <tt>TestCmd</tt> in
|
|
future.</p>
|
|
|
|
<p>Unfortunately, there are not documentation for <tt>TestCmd</tt>, apart
|
|
from the comments in code, so the most essential base methods will be
|
|
described here.</p>
|
|
|
|
<p>The basic steps of testing a build system behaviour are:</p>
|
|
|
|
<ol>
|
|
<li>Setting the initial working directory state</li>
|
|
|
|
<li>
|
|
Running the build system and checking:
|
|
|
|
<ol>
|
|
<li>generated output,</li>
|
|
|
|
<li>changes made to the working directory,</li>
|
|
|
|
<li>new content of the working directory.</li>
|
|
</ol>
|
|
</li>
|
|
|
|
<li>Adding, removing or touching files, or changing their content and
|
|
then repeating the previous step, until satisfied.</li>
|
|
</ol>
|
|
|
|
<p>Prior to using the test system, make sure you have checked out the
|
|
Boost CVS and have successfully build Jam -- which means that
|
|
<tt>bjam</tt> executable is available.</p>
|
|
|
|
<p>For example, suppose that you've placed under
|
|
<tt>tools/build/test</tt> a file <tt>test1.py</tt> and a subdirectory
|
|
<tt>test1</tt> with files <tt>foo.cpp</tt>, <tt>Jamfile</tt> and
|
|
<tt>Jamrules</tt>. A definition of function <tt>main</tt> is the only
|
|
content of <tt>foo.cpp</tt>, <tt>Jamfile</tt> builds an executable
|
|
<tt>foo</tt> from <tt>foo.cpp</tt> and <tt>Jamrules</tt> is empty.
|
|
Finally, <tt>test1.py</tt> contains:</p>
|
|
<pre class="code">
|
|
from BoostBuild import Tester, List
|
|
|
|
# Create a temporary working directory
|
|
t = Tester()
|
|
|
|
# Make content of the working directory equal to the content of the 'test1' directory.
|
|
t.set_tree('test1')
|
|
|
|
# Invoke 'bjam -sTOOLS=borland' and record which changes were made
|
|
t.run_build_system("-sTOOLS=borland")
|
|
# First, create a list of three pathnames
|
|
file_list = List("bin/foo/borland/debug/runtime-link-dynamic/foo") * List(".exe .obj")
|
|
# Second, assert that those files were added as result of the last build system invocation.
|
|
t.expect_addition(file_list)
|
|
|
|
# Invoke the build system once again
|
|
t.run_build_system("clean")
|
|
# Check if the files added previously were removed.
|
|
t.expect_removal(t.mul("bin/foo/borland/debug/runtime-link-dynamic/foo", [".exe", ".obj", ".tds"]))
|
|
</pre>
|
|
|
|
<p>This is all needed for a minimal test (and to find a bug in the
|
|
borland toolset!). Running the <tt>test1.py</tt> gives the following
|
|
output:</p>
|
|
<pre class="output">
|
|
File bin/foo/borland/debug/runtime-link-dynamic/foo.tds not removed as expected
|
|
FAILED test of D:\MyDocu~1\Work\build\boost-build\boost-build -d0
|
|
at line 144 of TestBoostBuild.py (expect_removal)
|
|
from line 12 of test1.py
|
|
</pre>
|
|
|
|
<p>Overview of the most important methods of class
|
|
<tt>TestBoostBuild</tt> follows.</p>
|
|
|
|
<h3><a name="sec-intro-changing">Changing the working directory</a></h3>
|
|
|
|
<p>The class <tt>TestBoostBuild</tt> creates a temporary directory in its
|
|
constructor and changes to that directory. It can be modified by calling
|
|
these methods:</p>
|
|
|
|
<ul>
|
|
<li><tt>set_tree</tt> -- sets the content of the working directory to
|
|
be equal to the content of the specified directory.</li>
|
|
|
|
<li><tt>write</tt> -- sets the content of file in a working
|
|
directory</li>
|
|
|
|
<li><tt>touch</tt> -- changes the modification times of a file</li>
|
|
</ul>
|
|
|
|
<h3><a name="sec-intro-examining">Examining the working directory and
|
|
changing it</a></h3>
|
|
|
|
<p>The method <tt>read</tt>, inherited from the <tt>TestCmd</tt> class,
|
|
can be used to read any file in the working directory and check its
|
|
content. <tt>TestBoostBuild</tt> adds another method for tracking
|
|
changes. Whenever build system is run (via <tt>run_build_system</tt>),
|
|
the state of working dir before and after running is recorded. In
|
|
addition, difference between the two states -- i.e. lists of files that
|
|
were added, removed, modified or touched -- is stored in two member
|
|
variables, <tt>tree_difference</tt> and
|
|
<tt>unexpected_difference</tt>.</p>
|
|
|
|
<p>After than, the test author may specify that some change is expected,
|
|
for example, by calling <tt>expect_addition("foo")</tt>. This call will
|
|
check if the file was indeed added, and if so, will remove its name from
|
|
the list of added files in <tt>unexpected_difference</tt>. Likewise, it's
|
|
possible to specify that some changes are not interesting, for example a
|
|
call <tt>ignore("*.obj")</tt> will just remove every files with ".obj"
|
|
extension from <tt>unexpected_difference</tt>.</p>
|
|
|
|
<p>When test has finished with expectations and ignoring, the member
|
|
<tt>unexpected_difference</tt> will contain the list of all changes not
|
|
yet accounted for. It is possible to assure that this list is empty by
|
|
calling <tt>expect_nothing_more</tt> member function.</p>
|
|
|
|
<h3><a name="sec-intro-results">Test result</a></h3>
|
|
|
|
<p>Any of the <tt>expect*</tt> methods below will fail the test if the
|
|
expectation is not met. It is also possible to perform manually arbitrary
|
|
test and explicitly cause the test to either pass or fail. Ordinary
|
|
filesystem functions can be used to work with the directory tree. Methods
|
|
<tt>pass_test</tt> and <tt>fail_test</tt> are used to explicitly give the
|
|
test outcome.</p>
|
|
|
|
<p>Typically, after test termination, the working directory is erased. If
|
|
is possible to prevent it by setting environmental variables
|
|
<tt>PRESERVE</tt>, <tt>PRESERVE_PASS</tt> and <tt>PRESERVE_FAIL</tt>.
|
|
Non-zero value of the first makes the working directory preserved in all
|
|
cases, while the other variables apply only to specific outcomes. When a
|
|
directory is preserved, its name will be printed to the standard
|
|
output.</p>
|
|
|
|
<h2><a name="sec-reference">Reference documentation</a></h2>
|
|
|
|
<p>The test system is composed of class <tt>Tester</tt>, derived form
|
|
<tt>TestCmd.TestCmd</tt>, and helper class <tt>List</tt>. The methods of
|
|
<tt>Tester</tt>, and the class <tt>List</tt> are described below.</p>
|
|
|
|
<p>The documentation frequently refer to filename. In all cases, files
|
|
are specified in unix style: a sequence of components, separated by "/".
|
|
This is true on all platforms. In some contexts, a list of files is
|
|
allowed. In that case any object with sequence interface is allowed.</p>
|
|
|
|
<h3><a name="method-__init__">Method <tt>__init__(self, arguments='',
|
|
executable='bjam')</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<ol>
|
|
<li>Remembers the current working directory in member
|
|
<tt>original_workdir</tt>.</li>
|
|
|
|
<li>Determines the location of executable (<code>bjam</code> by
|
|
default) and build system files, assuming that the current directory is
|
|
<tt>tools/build/test</tt>. Formulates jam invocation command, which
|
|
will include explicit setting for <tt>BOOST_BUILD_PATH</tt> variable
|
|
and arguments passed to this methods, if any. This command will be used
|
|
by subsequent invocation of <a href=
|
|
"#method-run_build_system"><tt>run_build_system</tt></a>. Finally,
|
|
initializes the base class.</li>
|
|
|
|
<li>Changes current working dir to the temporary working directory
|
|
created by the base constructor.</li>
|
|
</ol>
|
|
|
|
<h3><a name="method-set_tree">Method <tt>set_tree(self,
|
|
tree_location)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Replaces the content of the current working directory with the content
|
|
of directory at <tt>tree_location</tt>. If <tt>tree_location</tt> is not
|
|
absolute pathname, it will be treated as relative to
|
|
<tt>self.original_workdir</tt>. This methods also explicitly makes the
|
|
copied files writeable.</p>
|
|
|
|
<h3><a name="method-write">Method <tt>write(self, name,
|
|
content)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Writes the specified content to the file given by <tt>name</tt> under
|
|
the temporary working directory. If the file already exists, it is
|
|
overwritten. Any required directories are automatically created.</p>
|
|
|
|
<h3><a name="method-copy">Method <tt>copy(self, src, dst)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Equvivalent to <tt>self.write(self.read(src), dst)</tt>.</p>
|
|
|
|
<h3><a name="method-touch">Method <tt>touch(self, names)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Sets the access and modification times for all files in <tt>names</tt>
|
|
to the current time. All the elements in <tt>names</tt> should be
|
|
relative paths.</p>
|
|
|
|
<h3><a name="method-run_build_system">Method <tt>run_build_system(self,
|
|
subdir='', extra_args='', stdout=None, stderr='', status=0,
|
|
**kw)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<ol>
|
|
<li>Stores the state of the working directory in
|
|
<tt>self.previous_tree</tt>.</li>
|
|
|
|
<li>Changes to <tt>subdir</tt>, if it is specified. If it is not
|
|
absolute path, it is relative to the working dir.</li>
|
|
|
|
<li>Invokes the <tt>bjam</tt> executable, passing <tt>extra_args</tt>
|
|
to it. The binary should be located under
|
|
<tt><test_invocation_dir>/../jam_src/bin.<platform></tt>.
|
|
This is to make sure tests use the version of jam build from CVS.</li>
|
|
|
|
<li>Compares the stdout, stderr and exit status of build system
|
|
invocation with values to appropriate parameters, if they are not
|
|
<tt>None</tt>. If any difference is found, the test fails.</li>
|
|
|
|
<li>
|
|
<p>Stores the new state of the working directory in
|
|
<tt>self.tree</tt>. Computes the difference between previous and
|
|
current trees and store them in variables
|
|
<tt>self.tree_difference</tt> and
|
|
<tt>self.unexpected_difference</tt>.</p>
|
|
|
|
<p>Both variables are instances of class
|
|
<tt>tree.Trees_different</tt>, which have four attributes:
|
|
<tt>added_files</tt>, <tt>removed_files</tt>, <tt>modified_files</tt>
|
|
and <tt>touched_files</tt>. Each is a list of strings.</p>
|
|
</li>
|
|
</ol>
|
|
|
|
<h3><a name="method-read">Method <tt>read(self, name)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Read the specified file and returns it content. Raises an exception is
|
|
the file is absent.</p>
|
|
|
|
<h3><a name="method-read_and_strip">Method <tt>read_and_strip(self,
|
|
name)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Read the specified file and returns it content, after removing
|
|
trailing whitespace from every line. Raises an exception is the file is
|
|
absent.</p>
|
|
|
|
<p><b>Rationale:</b></p>
|
|
|
|
<p>Althought this method is questionable, there are a lot of cases when
|
|
jam or shells it uses insert spaces. It seems that introducing this
|
|
method is much simpler than dealing with all those cases.</p>
|
|
|
|
<h3><a name="methods-expectations">Methods for declaring
|
|
expectations</a></h3>
|
|
|
|
<p>Accordingly to the number of changes kinds that are detected, there
|
|
are four methods that specify that test author expects a specific change
|
|
to occur. They check <tt>self.unexpected_difference</tt>, and if the
|
|
change is present there, it is removed. Otherwise, test fails.</p>
|
|
|
|
<p>Each method accepts a list of names. Those names are always
|
|
unix-style, even on other systems.</p>
|
|
|
|
<p><b>Note:</b> The <tt>List</tt> helper class might be useful to create
|
|
lists of names.</p>
|
|
|
|
<p><b>Note:</b> The file content can be examined using
|
|
<tt>TestCmd.read</tt> function.</p>
|
|
|
|
<p>The members are:</p>
|
|
|
|
<ul>
|
|
<li>expect_addition</li>
|
|
|
|
<li>expect_removal</li>
|
|
|
|
<li>expect_modification</li>
|
|
|
|
<li>expect_touch</li>
|
|
|
|
<li>expect_nothing</li>
|
|
</ul>
|
|
|
|
<p>There's also a member <tt>expect_nothing_more</tt>, which checks that
|
|
all the changes are either expected or ignored, in other words that
|
|
<tt>unexpected_difference</tt> is empty by now.</p>
|
|
|
|
<p>Lastly, there's a method to compare file content with expected
|
|
content:</p>
|
|
<tt>expect_content(self, name, content, exact=0)</tt>
|
|
|
|
<p>The method fails the test if the content of file identified by 'name'
|
|
is different from 'content'. If 'exact' is true, the file content is used
|
|
as-is, otherwise, two transformations are applied:</p>
|
|
|
|
<ul>
|
|
<li>The <tt>read_and_strip</tt> method is used to read the file, which
|
|
removes trailing whitespace</li>
|
|
|
|
<li>Each backslash in the file content is converted to forward
|
|
slash.</li>
|
|
</ul>
|
|
|
|
<h3><a name="methods-ignoring">Methods for ignoring changes</a></h3>
|
|
|
|
<p>There are five methods which ignore changes made to the working tree.
|
|
They silently remove elements from <tt>self.unexpected_difference</tt>,
|
|
and don't generate error if element is not found. They accept shell style
|
|
wildcard.</p>
|
|
|
|
<p>The following methods correspond to four kinds of changes:</p>
|
|
|
|
<ul>
|
|
<li>ignore_addition(self, wildcard)</li>
|
|
|
|
<li>ignore_removal(self, wildcard)</li>
|
|
|
|
<li>ignore_modification(self, wildcard)</li>
|
|
|
|
<li>ignore_touch(self, wilcard)</li>
|
|
</ul>
|
|
|
|
<p>The method <tt>ignore(self, wildcard)</tt> ignores all the changes
|
|
made to files that match a wildcard.</p>
|
|
|
|
<h3><a name="methods-result">Methods for explicitly specifying
|
|
results</a></h3>
|
|
|
|
<h4>Method <tt>pass_test(self, condition=1)</tt></h4>
|
|
|
|
<div class="attention">
|
|
At this moment, the method should not be used.
|
|
</div>
|
|
|
|
<h4>Method <tt>fail_test(self, condition=1)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Cause the test to fail if <tt>condition</tt> is
|
|
true.</p>
|
|
|
|
<h3><a name="class-list">Helper class <tt>List</tt></a></h3>
|
|
The class has sequence interface and two additional methods.
|
|
|
|
<h4>Method <tt>__init__(self, string)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Splits the string on unescaped spaces and tabs. The
|
|
split components can further be retrieved using standard sequence
|
|
access.</p>
|
|
|
|
<h4>Method <tt>__mul__(self, other)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Returns an <tt>List</tt> instance, which elements are
|
|
all possible concatenations of two string, first of which is from
|
|
<tt>self</tt>, and second of which is from <tt>other</tt>.</p>
|
|
|
|
<p>The class also defines <tt>__str__</tt> and <tt>__repr__</tt> methods.
|
|
Finally, there's <tt>__coerce__</tt> method which allows to convert
|
|
strings to instances of <tt>List</tt>.</p>
|
|
|
|
<p><b>Example:</b></p>
|
|
<pre>
|
|
l = "a b" * List("c d")
|
|
for e in l:
|
|
print e
|
|
|
|
</pre>
|
|
|
|
<p>will output</p>
|
|
<pre>
|
|
ac
|
|
ad
|
|
bc
|
|
bd
|
|
|
|
</pre>
|
|
<hr>
|
|
|
|
<p class="revision">Last modified: May 5, 2002</p>
|
|
|
|
<p>© Copyright Vladimir Prus 2002. Permission to copy, use, modify,
|
|
sell and distribute this document is granted provided this copyright
|
|
notice appears in all copies. This document is provided ``as is'' without
|
|
express or implied warranty, and with no claim as to its suitability for
|
|
any purpose.</p>
|
|
</body>
|
|
</html>
|
|
|