- added new option for generating bindings: --multiple

- some refactoring of the code
- now detects forward declarations and prints a warning about them


[SVN r18187]

pyste/NEWS

@@ -1,3 +1,9 @@
+05 Apr 2003
+New option for generating the bindings: --multiple.
+
+02 Apr 2003
+Forward declarations are now detected and a warning is generated.
+
 24 Mar 2003
 Default policy for functions/methods that return const T& is now
 return_value_policy<copy_const_reference>().

pyste/doc/pyste.txt

@@ -72,26 +72,44 @@ Well, now let's fire it up:
 '''
 >python pyste.py
 
+Pyste version 0.6.5
+
 Usage:
     pyste [options] --module=<name> interface-files
 
 where options are:
-    -I <path>           add an include path
-    -D <symbol>         define symbol
+    -I <path>           add an include path    
+    -D <symbol>         define symbol    
+    --multiple          create various cpps, instead of only one 
+                        (useful during development)                        
+    --out               specify output filename (default: <module>.cpp)
+                        in --multiple mode, this will be a directory
     --no-using          do not declare "using namespace boost";
                         use explicit declarations instead
     --pyste-ns=<name>   set the namespace where new types will be declared;
                         default is "pyste"
+    --debug             writes the xml for each file parsed in the current
+                        directory
     -h, --help          print this help and exit
-    -v, --version       print version information
+    -v, --version       print version information      
 '''                        
 ]
 
 Options explained:
 
-The [^-I] and [^-D] are preprocessor flags, which are needed by gccxml to parse the header files correctly and by pyste to find the header files declared in the
+The [^-I] and [^-D] are preprocessor flags, which are needed by gccxml to parse
+the header files correctly and by pyste to find the header files declared in the
 interface files.
 
+[^--multiple] tells pyste to generate multiple cpps for this module (one for
+each header parsed) in the directory named by [^--out], instead of the usual
+single cpp file. This mode is useful during development of a binding, because
+you are constantly changing source files, re-generating the bindings and
+recompiling. This saves a lot of time in compiling.
+
+[^--out] names the output file (default: [^<module>.cpp]), or in multiple mode,
+names a output directory for the files (default: [^<module>]).
+
 [^--no-using] tells pyste to don't declare "[^using namespace boost;]" in the
 generated cpp, using the namespace boost::python explicitly in all declarations.
 Use only if you're having a name conflict in one of the files.
@@ -100,13 +118,24 @@ Use [^--pyste-ns] to change the namespace where new types are declared (for
 instance, the virtual wrappers). Use only if you are having any problems. By
 default, Pyste uses the empty namespace.
 
+[^--debug] will write in the current directory a xml file as outputted by gccxml
+for each header parsed. Useful for bug reports.
+
+[^-h, --help, -v, --version] are self-explaining, I believe. ;)
+
 So, the usage is simple enough:
 
 [pre >python pyste.py --module=mymodule file.pyste file2.pyste ...]
 
 will generate a file [^mymodule.cpp] in the same dir where the command was
 executed. Now you can compile the file using the same instructions of the
-[@../../doc/tutorial/doc/building_hello_world.html tutorial]. 
+[@../../doc/tutorial/doc/building_hello_world.html tutorial]. Or, if you prefer:
+
+[pre >python pyste.py --module=mymodule --multiple file.pyste file2.pyste ...]
+
+will create a directory named "mymodule" in the current directory, and will
+generate a bunch of cpp files, one for each header exported. You can then
+compile them all into a single shared library (or dll).
 
 [h2 Wait... how do I set those I and D flags?]
 

pyste/doc/running_pyste.html

@@ -47,25 +47,43 @@ Well, now let's fire it up:</p>
 
 &gt;python pyste.py
 
+Pyste version 0.6.5
+
 Usage:
     pyste [options] --module=&lt;name&gt; interface-files
 
 where options are:
-    -I &lt;path&gt;           add an include path
-    -D &lt;symbol&gt;         define symbol
+    -I &lt;path&gt;           add an include path    
+    -D &lt;symbol&gt;         define symbol    
+    --multiple          create various cpps, instead of only one 
+                        (useful during development)                        
+    --out               specify output filename (default: &lt;module&gt;.cpp)
+                        in --multiple mode, this will be a directory
     --no-using          do not declare &quot;using namespace boost&quot;;
                         use explicit declarations instead
     --pyste-ns=&lt;name&gt;   set the namespace where new types will be declared;
                         default is &quot;pyste&quot;
+    --debug             writes the xml for each file parsed in the current
+                        directory
     -h, --help          print this help and exit
-    -v, --version       print version information
+    -v, --version       print version information      
                         
 </pre></code><p>
 Options explained:</p>
 <p>
-The <tt>-I</tt> and <tt>-D</tt> are preprocessor flags, which are needed by gccxml to parse the header files correctly and by pyste to find the header files declared in the
+The <tt>-I</tt> and <tt>-D</tt> are preprocessor flags, which are needed by gccxml to parse
+the header files correctly and by pyste to find the header files declared in the
 interface files.</p>
 <p>
+<tt>--multiple</tt> tells pyste to generate multiple cpps for this module (one for
+each header parsed) in the directory named by <tt>--out</tt>, instead of the usual
+single cpp file. This mode is useful during development of a binding, because
+you are constantly changing source files, re-generating the bindings and
+recompiling. This saves a lot of time in compiling.</p>
+<p>
+<tt>--out</tt> names the output file (default: <tt>&lt;module&gt;.cpp</tt>), or in multiple mode,
+names a output directory for the files (default: <tt>&lt;module&gt;</tt>).</p>
+<p>
 <tt>--no-using</tt> tells pyste to don't declare &quot;<tt>using namespace boost;</tt>&quot; in the
 generated cpp, using the namespace boost::python explicitly in all declarations.
 Use only if you're having a name conflict in one of the files.</p>
@@ -74,12 +92,21 @@ Use <tt>--pyste-ns</tt> to change the namespace where new types are declared (fo
 instance, the virtual wrappers). Use only if you are having any problems. By
 default, Pyste uses the empty namespace.</p>
 <p>
+<tt>--debug</tt> will write in the current directory a xml file as outputted by gccxml
+for each header parsed. Useful for bug reports.</p>
+<p>
+<tt>-h, --help, -v, --version</tt> are self-explaining, I believe. ;)</p>
+<p>
 So, the usage is simple enough:</p>
 <code><pre>&gt;python pyste.py --module=mymodule file.pyste file2.pyste ...</pre></code><p>
 will generate a file <tt>mymodule.cpp</tt> in the same dir where the command was
 executed. Now you can compile the file using the same instructions of the
 <a href="../../doc/tutorial/doc/building_hello_world.html">
-tutorial</a>. </p>
+tutorial</a>. Or, if you prefer:</p>
+<code><pre>&gt;python pyste.py --module=mymodule --multiple file.pyste file2.pyste ...</pre></code><p>
+will create a directory named &quot;mymodule&quot; in the current directory, and will
+generate a bunch of cpp files, one for each header exported. You can then
+compile them all into a single shared library (or dll).</p>
 <a name="wait____how_do_i_set_those_i_and_d_flags_"></a><h2>Wait... how do I set those I and D flags?</h2><p>
 Don't worry: normally <a href="http://www.gccxml.org">
 GCCXML</a> is already configured correctly for your plataform, 

pyste/example/.cvsignore

@@ -1 +1,2 @@
 *.cpp
+.sconsign

pyste/src/ClassExporter.py

@@ -1,11 +1,10 @@
 import exporters
 from Exporter import Exporter
 from declarations import *
-from enumerate import enumerate
 from settings import *
-from CodeUnit import CodeUnit
+from SingleCodeUnit import SingleCodeUnit
 from EnumExporter import EnumExporter
-from makeid import makeid
+from utils import makeid, enumerate
 from copy import deepcopy
 import exporterutils
 import re
@@ -77,15 +76,16 @@ class ClassExporter(Exporter):
         return bases
 
     
-    def Order(self):
+    def ID(self):
         '''Return the TOTAL number of bases that this class has, including the
         bases' bases.  Do this because base classes must be instantialized
         before the derived classes in the module definition.  
         '''
-        return len(self.ClassBases())
+        return '%s_%s' % (len(self.ClassBases()), self.class_.FullName())
         
     
     def Export(self, codeunit, exported_names):
+        self.CheckForwardDeclarations()
         self.ExportBasics()
         self.ExportBases(exported_names)
         self.ExportConstructors()
@@ -99,6 +99,12 @@ class ClassExporter(Exporter):
         self.Write(codeunit)
 
 
+    def CheckForwardDeclarations(self):
+        for m in self.public_members:
+            if isinstance(m, Function):
+                exporterutils.WarnForwardDeclarations(m)
+
+    
     def Write(self, codeunit):
         indent = self.INDENT
         boost_ns = namespaces.python
@@ -522,7 +528,7 @@ class ClassExporter(Exporter):
             nested_info.name = nested_class.FullName()
             exporter = ClassExporter(nested_info)
             exporter.SetDeclarations(self.declarations + [nested_class])
-            codeunit = CodeUnit(None)
+            codeunit = SingleCodeUnit(None, None)
             exporter.Export(codeunit, exported_names)
             self.nested_codeunits.append(codeunit)
 
@@ -535,7 +541,7 @@ class ClassExporter(Exporter):
             enum_info.name = enum.FullName()
             exporter = EnumExporter(enum_info)
             exporter.SetDeclarations(self.declarations + [enum])
-            codeunit = CodeUnit(None)
+            codeunit = SingleCodeUnit(None, None)
             exporter.Export(codeunit, None)
             self.nested_codeunits.append(codeunit)
 

pyste/src/EnumExporter.py

@@ -28,3 +28,7 @@ class EnumExporter(Exporter):
             code += in_indent + '.value("%s", %s)\n' % (rename, value_fullname)
         code += indent + ';\n\n'
         codeunit.Write('module', code)
+
+
+    def ID(self):
+        return self.info.name

pyste/src/Exporter.py

@@ -60,10 +60,12 @@ class Exporter:
         return decls[0]
 
 
-    def Order(self):
-        '''Returns a number that indicates to which order this exporter
-        belongs.  The exporters will be called from the lowest order to the
-        highest order.
-        This function will only be called after Parse has been called. 
+    def ID(self):
+        '''Returns a string that uniquely identifies this instance. All
+        exporters will be sorted by ID before being exported.
         '''
-        return None # don't care
+        raise NotImplementedError
+
+
+    def Unit(self):
+        return self.info.include

pyste/src/FunctionExporter.py

@@ -19,6 +19,7 @@ class FunctionExporter(Exporter):
         decls = self.GetDeclarations(self.info.name)
         for decl in decls:
             self.info.policy = exporterutils.HandlePolicy(decl, self.info.policy)
+            exporterutils.WarnForwardDeclarations(decl)
             self.ExportDeclaration(decl, len(decls) == 1, codeunit)
         self.GenerateOverloads(decls, codeunit)            
 
@@ -76,3 +77,6 @@ class FunctionExporter(Exporter):
         else:
             return ''
 
+
+    def ID(self):
+        return self.info.name

pyste/src/GCCXMLParser.py

@@ -105,6 +105,7 @@ class GCCXMLParser(object):
         else:
             res = Type(decl.FullName(), const)
             res.volatile = volatile
+            res.incomplete = decl.incomplete
         return res            
         
                 
@@ -221,6 +222,7 @@ class GCCXMLParser(object):
         bases = self.GetBases(element.get('bases'))        
         location = self.GetLocation(element.get('location'))
         context = self.GetDecl(element.get('context'))
+        incomplete = bool(element.get('incomplete', False))
         if isinstance(context, str): 
             class_ = Class(name, context, [], abstract, bases)
             self.AddDecl(class_)
@@ -232,6 +234,7 @@ class GCCXMLParser(object):
         # we have to add the declaration of the class before trying        
         # to parse its members, to avoid recursion.
         class_.location = location
+        class_.incomplete = incomplete
         self.Update(id, class_)       
         # now we can get the members
         class_.members = self.GetMembers(element.get('members'))
@@ -258,14 +261,14 @@ class GCCXMLParser(object):
     def ParseReferenceType(self, id, element):
         type_ = self.GetType(element.get('type'))
         expand = not isinstance(type_, FunctionType)
-        ref = ReferenceType(type_.name, type_.const, None, expand)
+        ref = ReferenceType(type_.name, type_.const, None, type_.incomplete, expand)
         self.Update(id, ref)
         
         
     def ParsePointerType(self, id, element):
         type_ = self.GetType(element.get('type'))
         expand = not isinstance(type_, FunctionType)
-        ref = PointerType(type_.name, type_.const, None, expand)
+        ref = PointerType(type_.name, type_.const, None, type_.incomplete, expand)
         self.Update(id, ref)
         
         

pyste/src/HeaderExporter.py

@@ -63,5 +63,8 @@ class HeaderExporter(Exporter):
         exporter.SetDeclarations(self.declarations)
         exporters.exporters.append(exporter)
 
+    
+    def ID(self):
+        return self.info.include
 
         

pyste/src/IncludeExporter.py

@@ -17,3 +17,8 @@ class IncludeExporter(Exporter):
     def Parse(self, parser):
         pass
 
+    def ID(self):
+        return self.info.include
+
+    def Unit(self):
+        return '__all__'    # include it in all generated cpps (multiple mode)

pyste/src/MultipleCodeUnit.py

@@ -0,0 +1,105 @@
+from SingleCodeUnit import SingleCodeUnit
+import os
+import utils
+from SmartFile import SmartFile
+
+
+#==============================================================================
+# MultipleCodeUnit
+#==============================================================================
+class MultipleCodeUnit(object):
+    '''
+    Represents a bunch of cpp files, where each cpp file represents a header
+    to be exported by pyste. Another header, named _main.cpp is created too.
+    '''
+
+    def __init__(self, modulename, outdir):
+        self.modulename = modulename
+        self.outdir = outdir
+        self.codeunits = {}  # maps from a header to a SingleCodeUnit
+        self.functions = []
+        self._current = None
+
+    
+    def _FunctionName(self, code_unit_name):
+        return '_Export_%s' % utils.makeid(code_unit_name)
+
+    def _FileName(self, code_unit_name):
+        filename = os.path.basename(code_unit_name)
+        filename = '_%s.cpp' % os.path.splitext(filename)[0] 
+        return os.path.join(self.outdir, filename)
+
+    
+    def SetCurrent(self, code_unit_name):
+        'Changes the current code unit'
+        try:
+            codeunit = self.codeunits[code_unit_name]
+        except KeyError:
+            filename = self._FileName(code_unit_name)
+            function_name = self._FunctionName(code_unit_name)
+            codeunit = SingleCodeUnit(None, filename)
+            codeunit.module_definition = 'void %s()' % function_name
+            self.codeunits[code_unit_name] = codeunit
+            if code_unit_name != '__all__':
+                self.functions.append(function_name)
+        self._current = codeunit
+
+
+    def Current(self):
+        return self._current
+
+    current = property(Current, SetCurrent)
+        
+            
+    def _CheckCurrent(self):
+        if self.current is None:
+            raise RuntimeError, "No current code unit to write to!" 
+        
+        
+    def Write(self, section, code):
+        self._CheckCurrent()
+        self.current.Write(section, code)
+
+
+    def Section(self, section):
+        self._CheckCurrent()
+        return self.current.Section(section)
+
+
+    def _CreateOutputDir(self):
+        try:
+            os.mkdir(self.outdir)
+        except OSError: pass # already created
+
+        
+    def Save(self):
+        # create the directory where all the files will go
+        self._CreateOutputDir();
+        # write all the codeunits, merging first the contents of
+        # the special code unit named __all__
+        __all__ = self.codeunits.get('__all__')
+        for name, codeunit in self.codeunits.items():
+            if name != '__all__':                
+                if __all__:
+                    codeunit.Merge(__all__)
+                codeunit.Save()
+        # generate the main cpp
+        filename = os.path.join(self.outdir, '_main.cpp')
+        fout = SmartFile(filename, 'w')
+        fout.write(utils.left_equals('Include'))
+        fout.write('#include <boost/python.hpp>\n\n')
+        fout.write(utils.left_equals('Exports'))
+        for function in self.functions:
+            fout.write('void %s();\n' % function)
+        fout.write('\n')
+        fout.write(utils.left_equals('Module'))
+        fout.write('BOOST_PYTHON_MODULE(%s)\n' % self.modulename)
+        fout.write('{\n')
+        indent = ' ' * 4
+        for function in self.functions:
+            fout.write(indent)
+            fout.write('%s();\n' % function)
+        fout.write('}')
+        
+    
+        

pyste/src/SingleCodeUnit.py

@@ -1,18 +1,13 @@
-from settings import *
-
-#==============================================================================
-# RemoveDuplicatedLines
-#==============================================================================
-def RemoveDuplicatedLines(text):
-    includes = text.splitlines()
-    d = dict([(include, 0) for include in includes])
-    return '\n'.join(d.keys())
+from settings import namespaces
+import settings
+from utils import remove_duplicated_lines, left_equals
+from SmartFile import SmartFile
 
 
 #==============================================================================
-# CodeUnit
+# SingleCodeUnit
 #==============================================================================
-class CodeUnit:
+class SingleCodeUnit:
     '''
     Represents a cpp file, where other objects can write in one of the     
     predefined sections.
@@ -22,15 +17,16 @@ class CodeUnit:
         module - Inside the BOOST_PYTHON_MODULE macro
     '''
     
-    USING_BOOST_NS = True
-    
-    def __init__(self, modulename):
+    def __init__(self, modulename, filename):
         self.modulename = modulename
+        self.filename = filename
         # define the avaiable sections
         self.code = {}
         self.code['include'] = ''
         self.code['declaration'] = ''
         self.code['module'] = ''
+        # create the default module definition
+        self.module_definition = 'BOOST_PYTHON_MODULE(%s)' % modulename
 
 
     def Write(self, section, code):
@@ -39,36 +35,42 @@ class CodeUnit:
             raise RuntimeError, 'Invalid CodeUnit section: %s' % section
         self.code[section] += code
         
+
+    def Merge(self, other):
+        for section in ('include', 'declaration', 'module'):
+            self.code[section] = self.code[section] + other.code[section]    
+
         
     def Section(self, section):
         return self.code[section]
 
     
-    def Save(self, filename):
+    def Save(self):
         'Writes this code unit to the filename'
         space = '\n\n'
-        fout = file(filename, 'w')
+        fout = SmartFile(self.filename, 'w')
         # includes
-        includes = RemoveDuplicatedLines(self.code['include'])
-        fout.write('\n' + self._leftEquals('Includes'))        
+        includes = remove_duplicated_lines(self.code['include'])
+        fout.write('\n' + left_equals('Includes'))        
         fout.write('#include <boost/python.hpp>\n')
         fout.write(includes)
         fout.write(space)
         # using
-        if self.USING_BOOST_NS:
-            fout.write(self._leftEquals('Using'))
+        if settings.USING_BOOST_NS:
+            fout.write(left_equals('Using'))
             fout.write('using namespace boost::python;\n\n')
         # declarations
         if self.code['declaration']:
             pyste_namespace = namespaces.pyste[:-2]
-            fout.write(self._leftEquals('Declarations'))
+            fout.write(left_equals('Declarations'))
             fout.write('namespace %s {\n\n\n' % pyste_namespace)
             fout.write(self.code['declaration']) 
             fout.write('\n\n}// namespace %s\n' % pyste_namespace)
             fout.write(space)
         # module
-        fout.write(self._leftEquals('Module'))
-        fout.write('BOOST_PYTHON_MODULE(%s)\n{\n' % self.modulename)
+        fout.write(left_equals('Module'))
+        fout.write(self.module_definition + '\n')
+        fout.write('{\n')
         fout.write(self.code['module']) 
         fout.write('}\n')
 

pyste/src/SmartFile.py

@@ -0,0 +1,55 @@
+import os
+import md5
+
+#==============================================================================
+# SmartFile
+#==============================================================================
+class SmartFile(object):
+    '''
+    A file-like object used for writing files. The given file will only be
+    actually written to disk if there's not a file with the same name, or if
+    the existing file is *different* from the file to be written.
+    '''
+
+    def __init__(self, filename, mode='w'):
+        self.filename = filename
+        self.mode = mode
+        self._contents = []
+        self._closed = False
+
+
+    def __del__(self):
+        if not self._closed:
+            self.close()
+
+
+    def write(self, string):
+        self._contents.append(string)
+
+
+    def _dowrite(self, contents):
+        f = file(self.filename, self.mode)
+        f.write(contents)
+        f.close()
+        
+        
+    def _GetMD5(self, string):
+        return md5.new(string).digest()
+        
+    
+    def close(self):
+        # if the filename doesn't exist, write the file right away
+        this_contents = ''.join(self._contents)
+        if not os.path.isfile(self.filename):
+            self._dowrite(this_contents)
+        else:
+            # read the contents of the file already in disk
+            f = file(self.filename)
+            other_contents = f.read()
+            f.close()
+            # test the md5 for both files 
+            this_md5 = self._GetMD5(this_contents)
+            other_md5 = self._GetMD5(other_contents)
+            if this_md5 != other_md5:
+                self._dowrite(this_contents)
+        self._closed = True

pyste/src/declarations.py

@@ -15,6 +15,9 @@ class Declaration(object):
         self.namespace = namespace
         # tuple (filename, line)
         self.location = '', -1
+        # if a declaration is incomplete it means that it was
+        # forward declared
+        self.incomplete = False 
 
 
     def FullName(self):
@@ -265,13 +268,14 @@ class ConverterOperator(ClassOperator):
 class Type(Declaration):
     'Represents a type.'
 
-    def __init__(self, name, const=False, default=None):
+    def __init__(self, name, const=False, default=None, incomplete=False):
         Declaration.__init__(self, name, None)
         # whatever the type is constant or not
         self.const = const
         # used when the Type is a function argument
         self.default = default
         self.volatile = False
+        self.incomplete = incomplete
 
     def __repr__(self):
         if self.const:
@@ -304,8 +308,8 @@ class ArrayType(Type):
 class ReferenceType(Type): 
     'A reference type.'    
 
-    def __init__(self, name, const=False, default=None, expandRef=True):
-        Type.__init__(self, name, const, default)
+    def __init__(self, name, const=False, default=None, incomplete=False, expandRef=True):
+        Type.__init__(self, name, const, default, incomplete)
         self.expand = expandRef
         
         
@@ -321,8 +325,8 @@ class ReferenceType(Type):
 class PointerType(Type):
     'A pointer type.'
     
-    def __init__(self, name, const=False, default=None, expandPointer=False):
-        Type.__init__(self, name, const, default)
+    def __init__(self, name, const=False, default=None, incomplete=False, expandPointer=False):
+        Type.__init__(self, name, const, default, incomplete)
         self.expand = expandPointer
 
    

pyste/src/enumerate.py

@@ -1,7 +0,0 @@
-from __future__ import generators
-
-def enumerate(seq):
-    i = 0
-    for x in seq:
-        yield i, x
-        i += 1

pyste/src/exporterutils.py

@@ -28,6 +28,8 @@ class FunctionWrapper(object):
             return self.name
 
 
+_printed_warnings = {} # used to avoid double-prints of warnings
+
 #==============================================================================
 # HandlePolicy
 #==============================================================================
@@ -65,4 +67,23 @@ def HandlePolicy(function, policy):
             _printed_warnings[warning] = 1
     return policy
             
-_printed_warnings = {} # used to avoid double-prints in HandlePolicy
+
+
+#==============================================================================
+# WarnForwardDeclarations
+#==============================================================================
+def WarnForwardDeclarations(function):
+    '''Checks if any of the parameters or the result of the function are
+    incomplete types.'''
+
+    types = [function.result] + function.parameters
+    types = [x for x in types if x]
+    for type in types:
+        if type.incomplete:
+            msg = '---> Error: %s is forward declared. Please include the ' \
+                'appropriate header with its definition' % type.name
+            if msg not in _printed_warnings:
+                print msg
+                print 
+                _printed_warnings[msg] = 1
+    

pyste/src/infos.py

@@ -7,7 +7,7 @@ from IncludeExporter import IncludeExporter
 from EnumExporter import EnumExporter
 from HeaderExporter import HeaderExporter
 from exporterutils import FunctionWrapper
-from makeid import makeid
+from utils import makeid
 
 
 #==============================================================================

pyste/src/makeid.py

@@ -1,9 +0,0 @@
-
-def makeid(name):
-    'Returns the name as a valid identifier'
-    for invalidchar in ('::', '<', '>', ' ', ','):
-        name = name.replace(invalidchar, '_') 
-    # avoid duplications of '_' chars
-    names = [x for x in name.split('_') if x]
-    return '_'.join(names)
-

pyste/src/policies.py

@@ -73,3 +73,4 @@ reference_existing_object = 'reference_existing_object'
 copy_const_reference = 'copy_const_reference'
 copy_non_const_reference = 'copy_non_const_reference'
 manage_new_object = 'manage_new_object'        
+return_opaque_pointer = 'return_opaque_pointer'

pyste/src/pyste.py

@@ -5,12 +5,18 @@ Usage:
     pyste [options] --module=<name> interface-files
 
 where options are:
-    -I <path>           add an include path
-    -D <symbol>         define symbol
+    -I <path>           add an include path    
+    -D <symbol>         define symbol    
+    --multiple          create various cpps, instead of only one 
+                        (useful during development)                        
+    --out               specify output filename (default: <module>.cpp)
+                        in --multiple mode, this will be a directory
     --no-using          do not declare "using namespace boost";
                         use explicit declarations instead
     --pyste-ns=<name>   set the namespace where new types will be declared;
                         default is "pyste"
+    --debug             writes the xml for each file parsed in the current
+                        directory
     -h, --help          print this help and exit
     -v, --version       print version information                         
 '''
@@ -19,7 +25,8 @@ import sys
 import os
 import getopt
 import exporters
-import CodeUnit
+import SingleCodeUnit
+import MultipleCodeUnit
 import infos
 import exporterutils
 import settings
@@ -27,7 +34,7 @@ from policies import *
 from CppParser import CppParser, CppParserError
 import time
 
-__VERSION__ = '0.6.4'
+__VERSION__ = '0.6.5'
 
 def RecursiveIncludes(include):
     'Return a list containg the include dir and all its subdirectories'
@@ -58,7 +65,7 @@ def ParseArguments():
         options, files = getopt.getopt(
             sys.argv[1:], 
             'R:I:D:vh', 
-            ['module=', 'out=', 'no-using', 'pyste-ns=', 'debug', 'version', 'help'])
+            ['module=', 'multiple', 'out=', 'no-using', 'pyste-ns=', 'debug', 'version', 'help'])
     except getopt.GetoptError, e:
         print
         print 'ERROR:', e
@@ -67,6 +74,7 @@ def ParseArguments():
     defines = []
     module = None
     out = None
+    multiple = False
     for opt, value in options:
         if opt == '-I':
             includes.append(value)
@@ -80,11 +88,13 @@ def ParseArguments():
             out = value 
         elif opt == '--no-using':
             settings.namespaces.python = 'boost::python::'
-            CodeUnit.CodeUnit.USING_BOOST_NS = False
+            settings.USING_BOOST_NS = False
         elif opt == '--pyste-ns':
             settings.namespaces.pyste = value + '::'
         elif opt == '--debug':
             settings.DEBUG = True
+        elif opt == '--multiple':
+            multiple = True
         elif opt in ['-h', '--help']:
             Usage()
         elif opt in ['-v', '--version']:
@@ -95,10 +105,12 @@ def ParseArguments():
             Usage()
 
     if not files or not module:
-        Usage()
+        Usage()    
     if not out:
-        out = module + '.cpp'
-    return includes, defines, module, out, files
+        out = module
+        if not multiple:
+            out += '.cpp'
+    return includes, defines, module, out, files, multiple
 
     
 def CreateContext():
@@ -132,7 +144,7 @@ def CreateContext():
 
     
 def Main():
-    includes, defines, module, out, interfaces = ParseArguments()
+    includes, defines, module, out, interfaces, multiple = ParseArguments()
     # execute the interface files
     for interface in interfaces:
         context = CreateContext()
@@ -148,17 +160,22 @@ def Main():
             print '***', e, ': exitting'
             return 2
     print 
-    # sort the exporters by its order
-    exports = [(x.Order(), x) for x in exporters.exporters]
+    # sort the exporters by its ids
+    exports = [(x.ID(), x) for x in exporters.exporters]
     exports.sort()
     exports = [x for _, x in exports]
     # now generate the wrapper code
-    codeunit = CodeUnit.CodeUnit(module)
+    if multiple:
+        codeunit = MultipleCodeUnit.MultipleCodeUnit(module, out)
+    else:
+        codeunit = SingleCodeUnit.SingleCodeUnit(module, out)
     exported_names = []
     for export in exports:
+        if multiple:
+            codeunit.SetCurrent(export.Unit())
         export.GenerateCode(codeunit, exported_names)    
         exported_names.append(export.Name())
-    codeunit.Save(out)                
+    codeunit.Save()                
     print 'Module %s generated' % module
     return 0
 

pyste/src/settings.py

@@ -4,6 +4,7 @@
 #==============================================================================
 
 DEBUG = False
+USING_BOOST_NS = True
 
 class namespaces:
     boost = 'boost::'

pyste/src/utils.py

@@ -0,0 +1,39 @@
+from __future__ import generators
+
+#==============================================================================
+# enumerate
+#==============================================================================
+def enumerate(seq):
+    i = 0
+    for x in seq:
+        yield i, x
+        i += 1  
+
+
+#==============================================================================
+# makeid
+#==============================================================================
+def makeid(name):
+    'Returns the name as a valid identifier'
+    for invalidchar in ('::', '<', '>', ' ', ',', '.', '#'):
+        name = name.replace(invalidchar, '_') 
+    # avoid duplications of '_' chars
+    names = [x for x in name.split('_') if x]
+    return '_'.join(names)
+ 
+
+#==============================================================================
+# remove_duplicated_lines
+#==============================================================================
+def remove_duplicated_lines(text):
+    includes = text.splitlines()
+    d = dict([(include, 0) for include in includes])
+    return '\n'.join(d.keys())
+
+
+#==============================================================================
+# left_equals
+#==============================================================================
+def left_equals(s):
+        s = '// %s ' % s
+        return s + ('='*(80-len(s))) + '\n'  

pyste/tests/.cvsignore

@@ -1,3 +1,4 @@
 *.pyc
 *.dll
 *.cpp
+.sconsign

pyste/tests/SConstruct

@@ -0,0 +1,54 @@
+import glob
+import sys
+import os
+
+# constants
+if sys.platform == 'win32':
+    BOOST_ROOT = 'D:/Programming/Libraries/boost-cvs'
+    STLPORT_ROOT = 'D:/Programming/Libraries/stlport-4.5.3'
+    PYTHON_ROOT = 'C:/Python'
+
+if BOOST_ROOT:
+    BOOST_INCLUDE = BOOST_ROOT + '/boost'
+    BOOST_LIB = BOOST_ROOT + '/lib'
+
+if STLPORT_ROOT:
+    STLPORT_INCLUDE = STLPORT_ROOT + '/stlport'
+    STLPORT_LIB = STLPORT_ROOT + '/lib'
+
+if PYTHON_ROOT:
+    PYTHON_INCLUDE = PYTHON_ROOT + '/include'
+    PYTHON_LIB = PYTHON_ROOT + '/libs'
+
+LIBS = ['boost_python', 'python22']
+
+INCLUDES = ['../example']
+if sys.platform == 'win32':
+    CXX = 'icl'
+    CXXFLAGS='/GR /GX /MD /nologo'
+    INCLUDES += [BOOST_INCLUDE, STLPORT_INCLUDE, PYTHON_INCLUDE]
+    LIBPATH = [STLPORT_LIB, PYTHON_LIB, BOOST_LIB]
+else:
+    CXX = 'g++'
+    CXXFLAGS = ''
+    LIBPATH = []
+    #INCLUDES = ['..']
+
+# Create the environment
+env = Environment(
+    CXX=CXX,
+    CXXFLAGS=CXXFLAGS,
+    CPPPATH=INCLUDES,
+    LIBS=LIBS, 
+    LIBPATH=LIBPATH)
+
+
+# Build all the cpp files
+modules = [os.path.splitext(os.path.basename(x))[0] for x in glob.glob('../example/*.pyste')]
+for module in modules:
+    multiple = ARGUMENTS.get('multiple', '')
+    if multiple:
+        env.SharedLibrary(target=module, source=glob.glob(module+'/*.cpp'))
+    else:
+        env.SharedLibrary(target=module, source=module + '.cpp')
+

pyste/tests/SmartFileUT.py

@@ -0,0 +1,80 @@
+import sys
+sys.path.append('../src')
+from SmartFile import *
+import unittest
+import tempfile
+import os
+import time
+
+
+class SmartFileTest(unittest.TestCase):
+
+    FILENAME = tempfile.mktemp()
+    
+    def setUp(self):
+        self._Clean()
+        
+    def tearDown(self):
+        self._Clean()
+
+    def _Clean(self):
+        try:
+            os.remove(self.FILENAME)
+        except OSError: pass
+
+        
+    def testNonExistant(self):
+        "Must override the file, as there's no file in the disk yet"
+        self.assert_(not os.path.isfile(self.FILENAME))
+        f = SmartFile(self.FILENAME, 'w')
+        f.write('Testing 123\nTesting again.')
+        f.close()
+        self.assert_(os.path.isfile(self.FILENAME))
+
+
+    def testOverride(self):
+        "Must override the file, because the contents are different"
+        contents = 'Contents!\nContents!'
+        # create the file normally first
+        f = file(self.FILENAME, 'w')
+        f.write(contents)
+        f.close()
+        file_time = os.path.getmtime(self.FILENAME)
+        self.assert_(os.path.isfile(self.FILENAME))
+        time.sleep(2)
+        f = SmartFile(self.FILENAME, 'w')
+        f.write(contents + '_')
+        f.close()
+        new_file_time = os.path.getmtime(self.FILENAME)
+        self.assert_(new_file_time != file_time)
+
+
+    def testNoOverride(self):
+        "Must not override the file, because the contents are the same"
+        contents = 'Contents!\nContents!'
+        # create the file normally first
+        f = file(self.FILENAME, 'w')
+        f.write(contents)
+        f.close()
+        file_time = os.path.getmtime(self.FILENAME)
+        self.assert_(os.path.isfile(self.FILENAME))
+        time.sleep(2)
+        f = SmartFile(self.FILENAME, 'w')
+        f.write(contents)
+        f.close()
+        new_file_time = os.path.getmtime(self.FILENAME)
+        self.assert_(new_file_time == file_time) 
+
+
+    def testAutoClose(self):
+        "Must be closed when garbage-collected"
+        def foo():
+            f = SmartFile(self.FILENAME)
+            f.write('testing')  
+            self.assert_(not os.path.isfile(self.FILENAME))
+        foo()
+        self.assert_(os.path.isfile(self.FILENAME))
+        
+
+if __name__ == '__main__':
+    unittest.main()

pyste/tests/build_pyste_nt.bat

@@ -1,19 +0,0 @@
-@echo off
-setlocal
-set MODULE_NAME=%1
-set PYSTE_FILE=%2
-set BOOST_ROOT=d:/programming/libraries/boost-cvs
-set PYTHON_ROOT=c:/python
-set STLPORT_ROOT=d:/programming/libraries/stlport-4.5.3
-set PYSTE_FILE_DIR=%@PATH[%PYSTE_FILE]
-
-python ../src/pyste.py -I%PYSTE_FILE_DIR -I%BOOST_ROOT/boost --out=%MODULE_NAME.cpp --module=%MODULE_NAME %PYSTE_FILE
-
-icl /nologo /LD /GR /GX -I%PYSTE_FILE_DIR -I%STLPORT_ROOT/stlport -I%BOOST_ROOT/boost -I%PYTHON_ROOT/include %MODULE_NAME.cpp /link /libpath:%PYTHON_ROOT/libs /libpath:%BOOST_ROOT/lib /libpath:%STLPORT_ROOT/lib boost_python.lib
-
-rm %MODULE_NAME.cpp
-rm %MODULE_NAME.exp
-rm %MODULE_NAME.lib
-rm %MODULE_NAME.obj
-
-endlocal

pyste/tests/nt_all.bat

@@ -0,0 +1,7 @@
+@echo off
+call nt_build_all.bat
+runtests.py
+call nt_clean.bat
+call nt_build_all.bat --multiple
+runtests.py
+call nt_clean.bat 

pyste/tests/nt_build_all.bat

@@ -0,0 +1,13 @@
+@echo off
+
+call nt_build_pyste.bat basic %1
+call nt_build_pyste.bat enums %1 
+call nt_build_pyste.bat header_test %1 
+call nt_build_pyste.bat nested %1 
+call nt_build_pyste.bat operators %1 
+call nt_build_pyste.bat smart_ptr %1 
+call nt_build_pyste.bat templates %1 
+call nt_build_pyste.bat unions %1 
+call nt_build_pyste.bat virtual %1 
+call nt_build_pyste.bat virtual2 %1 
+call nt_build_pyste.bat wrappertest %1 

pyste/tests/nt_build_pyste.bat

@@ -0,0 +1,8 @@
+@echo off
+set BOOST_INCLUDE=D:\Programming\Libraries\boost-cvs\boost
+set out=%1.cpp
+if "%2" == "--multiple" set out=%1
+rem python ../src/pyste.py %2 -I%BOOST_INCLUDE -I../example --module=%1 --out=%out ../example/%1.pyste  
+pyste %2 -I%BOOST_INCLUDE -I../example --module=%1 --out=%out ../example/%1.pyste   
+
+scons --quiet multiple=%2 %1.dll

pyste/tests/nt_clean.bat

@@ -0,0 +1,21 @@
+@echo off
+
+rm -Rf basic
+rm -Rf enums
+rm -Rf header_test
+rm -Rf nested
+rm -Rf operators
+rm -Rf smart_ptr
+rm -Rf templates
+rm -Rf unions
+rm -Rf virtual
+rm -Rf virtual2
+rm -Rf wrappertest 
+
+rm -f *.cpp
+rm -f *.obj
+rm -f *.exp
+rm -f *.arg
+rm -f *.dll
+rm -f *.pyc
+rm -f *.lib 

pyste/tests/policiesUT.py

@@ -1,5 +1,5 @@
 import sys
-sys.path.append('..')  
+sys.path.append('../src')  
 import unittest
 from policies import *
 
@@ -35,7 +35,8 @@ class PoliciesTest(unittest.TestCase):
         self.assertEqual(x.Code(), ret % 'copy_non_const_reference') 
         x = return_value_policy(manage_new_object)
         self.assertEqual(x.Code(), ret % 'manage_new_object') 
-
+        x = return_value_policy(return_opaque_pointer)
+        self.assertEqual(x.Code(), ret % 'return_opaque_pointer') 
 
     def testReturnWithCustodiam(self):
         'test the mix of return_internal with custodian'

pyste/tests/test_all.bat

@@ -1,22 +0,0 @@
-@echo off
-call build_pyste_nt basic ../example/basic.pyste
-call build_pyste_nt enums ../example/enums.pyste
-call build_pyste_nt header_test ../example/header_test.pyste
-call build_pyste_nt nested ../example/nested.pyste
-call build_pyste_nt operators ../example/operators.pyste
-call build_pyste_nt templates ../example/templates.pyste
-call build_pyste_nt virtual ../example/virtual.pyste
-call build_pyste_nt wrappertest ../example/wrappertest.pyste
-call build_pyste_nt unions ../example/unions.pyste
-call build_pyste_nt virtual2 ../example/virtual2.pyste
-call build_pyste_nt smart_ptr ../example/smart_ptr.pyste
-
-
-runtests.py
-
-if errorlevel != 0 goto end
-
-rm *.dll
-rm *.pyc
-
-:end