mirror of
https://github.com/boostorg/build.git
synced 2026-02-01 08:22:15 +00:00
552 lines
19 KiB
HTML
552 lines
19 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
|
|
"http://www.w3.org/TR/html4/strict.dtd">
|
|
|
|
<html>
|
|
<head>
|
|
<meta name="generator" content=
|
|
"HTML Tidy for Linux/x86 (vers 1st April 2002), see www.w3.org">
|
|
<!--tidy options: -i -wrap 78 -->
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
|
|
<title>A testing system for Boost.Build</title>
|
|
|
|
<style type="text/css">
|
|
hr { color: black }
|
|
p.revision { text-align: right; font-style: italic }
|
|
pre.code { margin-left: 2em }
|
|
pre.example { margin-left: 2em; border: solid black thin }
|
|
pre.output { margin-left: 2em }
|
|
img.banner { border: 0; float: left }
|
|
h1 { text-align: right }
|
|
br.clear { clear: left }
|
|
div.attention { color: red }
|
|
</style>
|
|
</head>
|
|
|
|
<body>
|
|
<p><a href="../../../../index.htm"><img class="banner" height="86" width=
|
|
"277" alt="C++ Boost" src="../../../../boost.png"></a></p>
|
|
|
|
<h1>A testing system for Boost.Build<br class="clear">
|
|
</h1>
|
|
<hr>
|
|
|
|
<dl class="page-index">
|
|
<dt><a href="#sec-intro">Introduction for users</a></dt>
|
|
|
|
<dt><a href="#sec-developers">Introduction for developers</a></dt>
|
|
|
|
<dd>
|
|
<dl class="page-index">
|
|
<dt><a href="#sec-intro-changing">Changing the working
|
|
directory</a></dt>
|
|
|
|
<dt><a href="#sec-intro-examining">Examining the working directory
|
|
and changing it</a></dt>
|
|
|
|
<dt><a href="#sec-intro-results">Test result</a></dt>
|
|
</dl>
|
|
</dd>
|
|
|
|
<dt><a href="#sec-reference">Reference documentation</a></dt>
|
|
|
|
<dd>
|
|
<dl class="page-index">
|
|
<dt><a href="#method-__init__">Method __init__</a></dt>
|
|
|
|
<dt><a href="#method-set_tree">Method <tt>set_tree</tt></a></dt>
|
|
|
|
<dt><a href="#method-write">Method <tt>write</tt></a></dt>
|
|
|
|
<dt><a href="#method-copy">Method <tt>copy</tt></a></dt>
|
|
|
|
<dt><a href="#method-touch">Method <tt>touch</tt></a></dt>
|
|
|
|
<dt><a href="#method-run_build_system">Method
|
|
<tt>run_build_system</tt></a></dt>
|
|
|
|
<dt><a href="#method-read">Method <tt>read</tt></a></dt>
|
|
|
|
<dt><a href="#method-read_and_strip">Method
|
|
<tt>read_and_strip</tt></a></dt>
|
|
|
|
<dt><a href="#methods-expectations">Methods for declaring
|
|
expectations</a></dt>
|
|
|
|
<dt><a href="#methods-ignoring">Methods for ignoring
|
|
changes</a></dt>
|
|
|
|
<dt><a href="#methods-result">Methods for explicitly specifying
|
|
results</a></dt>
|
|
|
|
<dt><a href="#class-list">Helper class <tt>List</tt></a></dt>
|
|
</dl>
|
|
</dd>
|
|
</dl>
|
|
<hr>
|
|
|
|
<h2><a name="sec-intro">Introduction for users</a></h2>
|
|
|
|
<p>The testing system for Boost.Build is a small set of Python modules
|
|
and scripts for automatically testing user-obversable behaviour. It uses
|
|
components from testing systems of <a href=
|
|
"http://www.scons.org">Scons</a> and <a href=
|
|
"http://subversion.tigris.org">Subverion</a>, together with some
|
|
additional functionality.</p>
|
|
|
|
<p>To run the tests you'd need:</p>
|
|
|
|
<ol>
|
|
<li>Get the source tree of Boost.Build (located at <tt>tools/build</tt>
|
|
in Boost)</li>
|
|
|
|
<li>Have <a href="http://www.python.org">Python</a> installed. Version
|
|
2.1 is known to work.</li>
|
|
|
|
<li>Build Boost.Jam. See <a href=
|
|
"../../jam_src/index.html">$boost_build_root/jam_src/index.html</a> for
|
|
instructions.</li>
|
|
|
|
<li>Configure at least one toolset. You can edit
|
|
<tt>site-config.jam</tt> or <tt>user-config.jam</tt> to add new
|
|
toolsets. Or you can create file <tt>test-config.jam</tt> in
|
|
<tt>$boost_build_root/test</tt> directory. In this case,
|
|
<tt>site-config.jam</tt> and <tt>user-config.jam</tt> will be ignored
|
|
for testing.</li>
|
|
</ol>
|
|
|
|
<p>When all is done, you can run the tests with</p>
|
|
<pre class="code">
|
|
python test_all.py
|
|
</pre>
|
|
|
|
<p>which will use the default toolset, or you can specify toolset on the
|
|
command line, for example:</p>
|
|
<pre class="code">
|
|
python test_all.py borland
|
|
</pre>
|
|
|
|
<p>If everything's OK, you'll see a list of passed tests. Otherwise, a
|
|
failure will be reported.</p>
|
|
|
|
<p>It is possible to run a specific test, for example:</p>
|
|
<pre class="code">
|
|
python generators_test.py
|
|
</pre>
|
|
|
|
<h2><a name="sec-developers">Introduction for developers</a></h2>
|
|
|
|
<p>It is suggested that every new functionality come together with tests,
|
|
and that bugfixes are accompanied by tests. There's no need to say that
|
|
tests are good, but two points are extremely important:</p>
|
|
|
|
<ul>
|
|
<li>For an interpreted language like Jam, without any static checks,
|
|
testing is simply the only sefeguard we can have.</li>
|
|
|
|
<li>Good tests allow to change internal design much more safely, and we
|
|
didn't nailed everything down yet.</li>
|
|
</ul>
|
|
|
|
<p>Adding a new test is simple:</p>
|
|
|
|
<ol>
|
|
<li>Go to <tt>$boost_build_root/test/test_all.py</tt> and add new test
|
|
name to the list at the end of file. Suppose the test name is
|
|
"hello".</li>
|
|
|
|
<li>Add a new python module, in this example "hello.py", to do actual
|
|
testing.</li>
|
|
</ol>
|
|
|
|
<p>The module, in general will perform these basic actions:</p>
|
|
|
|
<ol>
|
|
<li>Setting the initial working directory state</li>
|
|
|
|
<li>
|
|
Running the build system and checking:
|
|
|
|
<ol>
|
|
<li>generated output,</li>
|
|
|
|
<li>changes made to the working directory,</li>
|
|
|
|
<li>new content of the working directory.</li>
|
|
</ol>
|
|
</li>
|
|
|
|
<li>Adding, removing or touching files, or changing their content and
|
|
then repeating the previous step, until satisfied.</li>
|
|
|
|
<li>Cleaning up</li>
|
|
</ol>
|
|
|
|
<p>The "hello.py" module might contain:</p>
|
|
<pre class="example">
|
|
from BoostBuild import Tester, List
|
|
|
|
# Create a temporary working directory
|
|
t = Tester()
|
|
|
|
# Create the needed files
|
|
t.write("project-root.jam", "")
|
|
t.write("Jamfile", """
|
|
exe hello : hello.cpp ;
|
|
""")
|
|
t.write("hello.cpp", """
|
|
int main()
|
|
{
|
|
return 0;
|
|
}
|
|
|
|
""")
|
|
|
|
t.run_build_system()
|
|
|
|
# First, create a list of three pathnames
|
|
file_list = List("bin/$toolset/debug/") * List("hello.exe hello.obj")
|
|
# Second, assert that those files were added as result of the last build system invocation.
|
|
t.expect_addition(file_list)
|
|
|
|
# Invoke the build system once again
|
|
t.run_build_system("clean")
|
|
# Check if the files added previously were removed.
|
|
t.expect_removal(file_list)
|
|
|
|
# Remove temporary directories
|
|
t.cleanup()
|
|
</pre>
|
|
|
|
<p>The <tt>test</tt> directory contains a file "template.py" which can be
|
|
used as a start for your own tests.</p>
|
|
|
|
<p>Overview of the most important methods of class
|
|
<tt>TestBoostBuild</tt> follows.</p>
|
|
|
|
<h3><a name="sec-intro-changing">Changing the working directory</a></h3>
|
|
|
|
<p>The class <tt>TestBoostBuild</tt> creates a temporary directory in its
|
|
constructor and changes to that directory. It can be modified by calling
|
|
these methods:</p>
|
|
|
|
<ul>
|
|
<li><tt>set_tree</tt> -- sets the content of the working directory to
|
|
be equal to the content of the specified directory. This method is
|
|
preferrable when directory tree for testing is large.</li>
|
|
|
|
<li><tt>write</tt> -- sets the content of file in a working directory.
|
|
This is optimal if you want to create a directory tree with 3-4 small
|
|
files.</li>
|
|
|
|
<li><tt>touch</tt> -- changes the modification times of a file</li>
|
|
</ul>
|
|
|
|
<h3><a name="sec-intro-examining">Examining the working directory and
|
|
changing it</a></h3>
|
|
|
|
<p>The method <tt>read</tt>, inherited from the <tt>TestCmd</tt> class,
|
|
can be used to read any file in the working directory and check its
|
|
content. <tt>TestBoostBuild</tt> adds another method for tracking
|
|
changes. Whenever build system is run (via <tt>run_build_system</tt>),
|
|
the state of working dir before and after running is recorded. In
|
|
addition, difference between the two states -- i.e. lists of files that
|
|
were added, removed, modified or touched -- is stored in two member
|
|
variables, <tt>tree_difference</tt> and
|
|
<tt>unexpected_difference</tt>.</p>
|
|
|
|
<p>After than, the test author may specify that some change is expected,
|
|
for example, by calling <tt>expect_addition("foo")</tt>. This call will
|
|
check if the file was indeed added, and if so, will remove its name from
|
|
the list of added files in <tt>unexpected_difference</tt>. Likewise, it's
|
|
possible to specify that some changes are not interesting, for example a
|
|
call <tt>ignore("*.obj")</tt> will just remove every files with ".obj"
|
|
extension from <tt>unexpected_difference</tt>.</p>
|
|
|
|
<p>When test has finished with expectations and ignoring, the member
|
|
<tt>unexpected_difference</tt> will contain the list of all changes not
|
|
yet accounted for. It is possible to assure that this list is empty by
|
|
calling <tt>expect_nothing_more</tt> member function.</p>
|
|
|
|
<h3><a name="sec-intro-results">Test result</a></h3>
|
|
|
|
<p>Any of the <tt>expect*</tt> methods below will fail the test if the
|
|
expectation is not met. It is also possible to perform manually arbitrary
|
|
test and explicitly cause the test to either pass or fail. Ordinary
|
|
filesystem functions can be used to work with the directory tree. Methods
|
|
<tt>pass_test</tt> and <tt>fail_test</tt> are used to explicitly give the
|
|
test outcome.</p>
|
|
|
|
<p>Typically, after test termination, the working directory is erased. To
|
|
debug a failed test, you should add "--preserve" option when invoking
|
|
test. On failure, the working directory will be copied to "failed_test"
|
|
directory in the current dir.</p>
|
|
|
|
<h2 id="sec-reference">Reference documentation</h2>
|
|
|
|
<p>The test system is composed of class <tt>Tester</tt>, derived form
|
|
<tt>TestCmd.TestCmd</tt>, and helper class <tt>List</tt>. The methods of
|
|
<tt>Tester</tt>, and the class <tt>List</tt> are described below.</p>
|
|
|
|
<p>The documentation frequently refer to filename. In all cases, files
|
|
are specified in unix style: a sequence of components, separated by "/".
|
|
This is true on all platforms. In some contexts, a list of files is
|
|
allowed. In that case any object with sequence interface is allowed.</p>
|
|
|
|
<h3><a name="method-__init__">Method <tt>__init__(self, arguments='',
|
|
executable='bjam')</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<ol>
|
|
<li>Remembers the current working directory in member
|
|
<tt>original_workdir</tt>.</li>
|
|
|
|
<li>Determines the location of executable (<code>bjam</code> by
|
|
default) and build system files, assuming that the current directory is
|
|
<tt>tools/build/test</tt>. Formulates jam invocation command, which
|
|
will include explicit setting for <tt>BOOST_BUILD_PATH</tt> variable
|
|
and arguments passed to this methods, if any. This command will be used
|
|
by subsequent invocation of <a href=
|
|
"#method-run_build_system"><tt>run_build_system</tt></a>. Finally,
|
|
initializes the base class.</li>
|
|
|
|
<li>Changes current working dir to the temporary working directory
|
|
created by the base constructor.</li>
|
|
</ol>
|
|
|
|
<h3><a name="method-set_tree">Method <tt>set_tree(self,
|
|
tree_location)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Replaces the content of the current working directory with the content
|
|
of directory at <tt>tree_location</tt>. If <tt>tree_location</tt> is not
|
|
absolute pathname, it will be treated as relative to
|
|
<tt>self.original_workdir</tt>. This methods also explicitly makes the
|
|
copied files writeable.</p>
|
|
|
|
<h3><a name="method-write">Method <tt>write(self, name,
|
|
content)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Writes the specified content to the file given by <tt>name</tt> under
|
|
the temporary working directory. If the file already exists, it is
|
|
overwritten. Any required directories are automatically created.</p>
|
|
|
|
<h3><a name="method-copy">Method <tt>copy(self, src, dst)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Equvivalent to <tt>self.write(self.read(src), dst)</tt>.</p>
|
|
|
|
<h3><a name="method-touch">Method <tt>touch(self, names)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Sets the access and modification times for all files in <tt>names</tt>
|
|
to the current time. All the elements in <tt>names</tt> should be
|
|
relative paths.</p>
|
|
|
|
<h3><a name="method-run_build_system">Method <tt>run_build_system(self,
|
|
subdir='', extra_args='', stdout=None, stderr='', status=0,
|
|
**kw)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<ol>
|
|
<li>Stores the state of the working directory in
|
|
<tt>self.previous_tree</tt>.</li>
|
|
|
|
<li>Changes to <tt>subdir</tt>, if it is specified. If it is not
|
|
absolute path, it is relative to the working dir.</li>
|
|
|
|
<li>Invokes the <tt>bjam</tt> executable, passing <tt>extra_args</tt>
|
|
to it. The binary should be located under
|
|
<tt><test_invocation_dir>/../jam_src/bin.<platform></tt>.
|
|
This is to make sure tests use the version of jam build from CVS.</li>
|
|
|
|
<li>Compares the stdout, stderr and exit status of build system
|
|
invocation with values to appropriate parameters, if they are not
|
|
<tt>None</tt>. If any difference is found, the test fails.</li>
|
|
|
|
<li>
|
|
<p>Stores the new state of the working directory in
|
|
<tt>self.tree</tt>. Computes the difference between previous and
|
|
current trees and store them in variables
|
|
<tt>self.tree_difference</tt> and
|
|
<tt>self.unexpected_difference</tt>.</p>
|
|
|
|
<p>Both variables are instances of class
|
|
<tt>tree.Trees_different</tt>, which have four attributes:
|
|
<tt>added_files</tt>, <tt>removed_files</tt>, <tt>modified_files</tt>
|
|
and <tt>touched_files</tt>. Each is a list of strings.</p>
|
|
</li>
|
|
</ol>
|
|
|
|
<h3><a name="method-read">Method <tt>read(self, name)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Read the specified file and returns it content. Raises an exception is
|
|
the file is absent.</p>
|
|
|
|
<h3><a name="method-read_and_strip">Method <tt>read_and_strip(self,
|
|
name)</tt></a></h3>
|
|
|
|
<p><b>Effects:</b></p>
|
|
|
|
<p>Read the specified file and returns it content, after removing
|
|
trailing whitespace from every line. Raises an exception is the file is
|
|
absent.</p>
|
|
|
|
<p><b>Rationale:</b></p>
|
|
|
|
<p>Althought this method is questionable, there are a lot of cases when
|
|
jam or shells it uses insert spaces. It seems that introducing this
|
|
method is much simpler than dealing with all those cases.</p>
|
|
|
|
<h3><a name="methods-expectations">Methods for declaring
|
|
expectations</a></h3>
|
|
|
|
<p>Accordingly to the number of changes kinds that are detected, there
|
|
are four methods that specify that test author expects a specific change
|
|
to occur. They check <tt>self.unexpected_difference</tt>, and if the
|
|
change is present there, it is removed. Otherwise, test fails.</p>
|
|
|
|
<p>Each method accepts a list of names. Those names use <tt>/</tt> path
|
|
separator on all systems. Additionaly, the test system translates
|
|
suffixes appropriately. For the test to be portable, suffixes should use
|
|
Windows convention: <tt>exe</tt> for executables, <tt>dll</tt> for
|
|
dynamic libraries and <tt>lib</tt> for static libraries. Lastly, the
|
|
string "$toolset" in file names is replaced by the name of tested
|
|
toolset.</p>
|
|
|
|
<p><b>Note:</b> The <tt>List</tt> helper class might be useful to create
|
|
lists of names.</p>
|
|
|
|
<p><b>Note:</b> The file content can be examined using
|
|
<tt>TestCmd.read</tt> function.</p>
|
|
|
|
<p>The members are:</p>
|
|
|
|
<ul>
|
|
<li>expect_addition</li>
|
|
|
|
<li>expect_removal</li>
|
|
|
|
<li>expect_modification</li>
|
|
|
|
<li>expect_nothing</li>
|
|
</ul>
|
|
|
|
<p>Note that <tt>expect_modification</tt> is used to check that a either
|
|
file content or timestamp has changed. The rationale is that some
|
|
compilers change content even if sources does not change, and it's easier
|
|
to have a method which checks for both content and time changes.</p>
|
|
|
|
<p>There's also a member <tt>expect_nothing_more</tt>, which checks that
|
|
all the changes are either expected or ignored, in other words that
|
|
<tt>unexpected_difference</tt> is empty by now.</p>
|
|
|
|
<p>Lastly, there's a method to compare file content with expected
|
|
content:</p>
|
|
<tt>expect_content(self, name, content, exact=0)</tt>
|
|
|
|
<p>The method fails the test if the content of file identified by 'name'
|
|
is different from 'content'. If 'exact' is true, the file content is used
|
|
as-is, otherwise, two transformations are applied:</p>
|
|
|
|
<ul>
|
|
<li>The <tt>read_and_strip</tt> method is used to read the file, which
|
|
removes trailing whitespace</li>
|
|
|
|
<li>Each backslash in the file content is converted to forward
|
|
slash.</li>
|
|
</ul>
|
|
|
|
<h3><a name="methods-ignoring">Methods for ignoring changes</a></h3>
|
|
|
|
<p>There are five methods which ignore changes made to the working tree.
|
|
They silently remove elements from <tt>self.unexpected_difference</tt>,
|
|
and don't generate error if element is not found. They accept shell style
|
|
wildcard.</p>
|
|
|
|
<p>The following methods correspond to four kinds of changes:</p>
|
|
|
|
<ul>
|
|
<li>ignore_addition(self, wildcard)</li>
|
|
|
|
<li>ignore_removal(self, wildcard)</li>
|
|
|
|
<li>ignore_modification(self, wildcard)</li>
|
|
|
|
<li>ignore_touch(self, wilcard)</li>
|
|
</ul>
|
|
|
|
<p>The method <tt>ignore(self, wildcard)</tt> ignores all the changes
|
|
made to files that match a wildcard.</p>
|
|
|
|
<h3><a name="methods-result">Methods for explicitly specifying
|
|
results</a></h3>
|
|
|
|
<h4>Method <tt>pass_test(self, condition=1)</tt></h4>
|
|
|
|
<div class="attention">
|
|
At this moment, the method should not be used.
|
|
</div>
|
|
|
|
<h4>Method <tt>fail_test(self, condition=1)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Cause the test to fail if <tt>condition</tt> is
|
|
true.</p>
|
|
|
|
<h3><a name="class-list">Helper class <tt>List</tt></a></h3>
|
|
The class has sequence interface and two additional methods.
|
|
|
|
<h4>Method <tt>__init__(self, string)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Splits the string on unescaped spaces and tabs. The
|
|
split components can further be retrieved using standard sequence
|
|
access.</p>
|
|
|
|
<h4>Method <tt>__mul__(self, other)</tt></h4>
|
|
|
|
<p><b>Effects:</b> Returns an <tt>List</tt> instance, which elements are
|
|
all possible concatenations of two string, first of which is from
|
|
<tt>self</tt>, and second of which is from <tt>other</tt>.</p>
|
|
|
|
<p>The class also defines <tt>__str__</tt> and <tt>__repr__</tt> methods.
|
|
Finally, there's <tt>__coerce__</tt> method which allows to convert
|
|
strings to instances of <tt>List</tt>.</p>
|
|
|
|
<p><b>Example:</b></p>
|
|
<pre>
|
|
l = "a b" * List("c d")
|
|
for e in l:
|
|
print e
|
|
|
|
</pre>
|
|
|
|
<p>will output</p>
|
|
<pre>
|
|
ac
|
|
ad
|
|
bc
|
|
bd
|
|
|
|
</pre>
|
|
<hr>
|
|
|
|
<p class="revision">Last modified: Mar 3, 2003</p>
|
|
|
|
<p>© Copyright Vladimir Prus 2002, 2003. Permission to copy, use,
|
|
modify, sell and distribute this document is granted provided this
|
|
copyright notice appears in all copies. This document is provided ``as
|
|
is'' without express or implied warranty, and with no claim as to its
|
|
suitability for any purpose.</p>
|
|
</body>
|
|
</html>
|
|
|