Dependency scanning is the process of finding implicit dependencies - due to "include" statements and similar things. It has to take into - account two things:
+Dependency scanning is the process of finding implicit dependencies, + like "#include" statements in C++. The requirements for right dependency + scanning mechanism are:
-
-
- Whether includes in a particular file need to be taken into account - depends on actions that use that file. For example, if the action is - "copy file", then includes should be ignored. Another example is when a - file is compiled with two different include paths on different - toolsets. +
- Support for different scanning algorithm. C++ and XML have quite + different syntax for includes and rules for looking up included + files. -
- It is possible to include generated header. In which case, it may - not yet exist at the time when we scan dependencies. +
- Ability to scan the same file several times. For example, single + C++ file can be compiled with different include paths. + +
- Proper detection of dependencies on generated files. + +
- Proper detection of dependencies from generated file.
Dependency scanning is implemented by objects called scanners. See - documentation for the "scanner" module to detail.
+Support for different scanning algorithm
-Regarding the first problem, we really have no choice. We can't treat - the same actual target differently depending on from where it is used. - Therefore, when handling of includes differers depending on actions, we - have to duplicate targets and assign different properties to it.
+Different scanning algorithm are encapsulated by objects called + "scanners". Please see the documentation for "scanner" module for more + details.
-For the reason, when actualizing a virtual target we optionally pass - the needed scanner to the "virtual-target.actualize" method. When no - scanner is passed, a new actual target is created, with it's dependencies - and updating actions set accordingly. When a particular scanner is - specified, a new actual target is created. That target will depend on - target created without scanner. In effect, this will allow to use - different scanners for the same file.
+Ability to scan the same file several times
-Generated sources
- Let me explain what I find the right semantic, first without any - subvariants. We have a target "a.cpp" which includes "a_parser.h", we - have to search through all include directories, checking: +As said above, it's possible to compile C++ file twice, with different + include path. Therefore, include dependencies for those compilation can + be different. The problem is that bjam does not allow several scans of + the same target.
+ +The solution in Boost.Build is straigtforward. When a virtual target + is converted to bjam target (via virtual-target.actualize + method), we specify the scanner object to be used. The actualize method + will create different bjam targets for different scanners.
+ +All targets with specific scanner are made dependent on target without + scanner, which target is always created. This is done in case target is + updated. The updating action will specify target without scanner and + output, and we need targets with scanner to be updated as well.
+ +For example, assume that "a.cpp" is compiled by two compilers with + different include path. It's also copied into some install location. In + turn, it's produced from "a.verbatim". The dependency graph will look + like:
++ a.o (<toolset>gcc) <--(compile)-- a.cpp (scanner1) ----+ + a.o (<toolset>msvc) <--(compile)-- a.cpp (scanner2) ----| + a.cpp (installed copy) <--(copy) ----------------------- a.cpp (no scanner) + ^ + | + a.verbose --------------------------------+ + ++ +
Proper detection of dependencies on generated files.
+ +This requirement breaks down to the following ones.
-
-
- If there's such file there, or +
- If when compiling "a.cpp" there's include of "a.h", the "dir" + directory is in include path, and a target called "a.h" will be + generated to "dir", then bjam should discover the include, and create + "a.h" before compiling "a.cpp". -
- If there's a target of the same name, bound to that dir via LOCATE - variable. +
- Since almost always Boost.Build generates targets to a "bin" + directory, it should be supported as well. I.e. in the scanario above, + Jamfile in "dir" might create a main target, which generates "a.h". The + file will be generated to "dir/bin" directory, but we still have to + recornize the dependency.
There are two design choices. Suppose we have files a.cpp and b.cpp, - and each one includes header.h, generated by some action. Dependency - graph created by classic jam would look like:
+The first requirement means that when determining what "a.h" means, + when found in "a.cpp", we have to iterate over all directories in include + paths, checking for each one:
+ +-
+
- If there's file "a.h" in that directory, or + +
- If there's a target called "a.h", which will be generated to that + directory. +
Classic Jam has built-in facilities for point (1) above, but that's + not enough. It's hard to implement the right semantic without builtin + support. For example, we could try to check if there's targer called + "a.h" somewhere in dependency graph, and add a dependency to it. The + problem is that without search in include path, the semantic may be + incorrect. For example, one can have an action which generated some + "dummy" header, for system which don't have the native one. Naturally, we + don't want to depend on that generated header on platforms where native + one is included.
+ +There are two design choices for builtin support. Suppose we have + files a.cpp and b.cpp, and each one includes header.h, generated by some + action. Dependency graph created by classic jam would look like:
a.cpp -----> <scanner1>header.h [search path: d1, d2, d3]
@@ -158,18 +199,49 @@
|
b.cpp -----> <scanner2>header.h [ search path: d1, d2, d4]
- The first alternative was use for some time. The problem however is: what
- include paths should be used when scanning header.h? Originally, two
- different sets of include paths were used. The second alternative does
- not have this problem, so it's implemented now.
+ The first alternative was used for some time. The problem however is:
+ what include paths should be used when scanning header.h? The second
+ alternative was suggested by Matt Armstrong. It has similiar effect: add
+ targets which depend on <scanner1>header.h will also depend on
+ <d2>header.h. But now we have two different target with two
+ different scanners, and those targets can be scanned independently. The
+ problem of first alternative is avoided, so the second alternative is
+ implemented now.
- Includes between generated sources
+The second sub-requirements is that targets generated to "bin" + directory are handled as well. Boost.Build implements semi-automatic + approach. When compiling C++ files the process is:
+ +-
+
- The main target to which compiled file belongs is found. + +
- All other main targets that the found one depends on are found. + Those include main target which are used as sources, or present as + values of "dependency" features. + +
- All directories where files belonging to those main target will be + generated are added to the include path. +
After this is done, dependencies are found by the approach explained + previously.
+ +Note that if a target uses generated headers from other main target, + that main target should be explicitly specified as dependency property. + It would be better to lift this requirement, but it seems not very + problematic in practice.
+ +For target types other than C++, adding of include paths must be + implemented anew.
+ +Proper detection of dependencies from generated files
Suppose file "a.cpp" includes "a.h" and both are generated by some - action. Initially, neither file exists, so when classic jam constructs - dependency graph, the include is not found. As the result, jam might - attempt to compile a.cpp before creating a.h, and compilation will - fail.
+ action. Note that classic jam has two stages. In first stage dependency + graph graph is build and actions which should be run are determined. In + second stage the actions are executed. Initially, neither file exists, so + the include is not found. As the result, jam might attempt to compile + a.cpp before creating a.h, and compilation will fail.